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Abstract 

Space  systems  are  a  critical  enabler  of  the  net-centric  operation  warfare  (NCOW) 
needed  to  achieve  victory  in  the  Global  War  on  Terrorism  (GWOT).  The  effective 
acquisition  of  affordable  systems  is  vital  to  our  National  Security  Strategy.  Space 
systems  play  an  important  role  throughout  a  wide  spectrum  of  military  and  civil 
operations.  Several  challenging  factors  unique  to  space  systems  development  are  the 
high  level  of  technological  complexity,  a  broad  joint  user  base,  and  the  reliance  on 
seamless  interoperable  systems  to  achieve  superior  capabilities  for  US  warfighters.  This 
research  examines  the  interaction  between  the  Joint  Capabilities  Integration  and 
Development  System  (JCIDS)  and  the  National  Security  Space  Acquisition  Process 
(NSSAP)  through  a  qualitative  case  study  and  identifies  ways  to  improve  this  interaction 
by  answering  investigative  questions  and  providing  recommendations  to  be  tested  in 
future  research. 
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REVIEW  OF  THE  JOINT  CAPABILITY  INTEGRATION  AND  DEVELOPMENT 
SYSTEM  (JCIDS)  AND  THE  NATIONAL  SECURITY  SPACE  ACQUISITION 

PROCESS  (NS SAP) 

I.  Introduction 

1.1  Space  System  Criticality 

Space  systems  are  a  critical  enabler  of  the  net-centric  operation  warfare  (NCOW) 
needed  to  achieve  victory  in  the  Global  War  on  Terrorism  (GWOT).  A  joint  mix  of 
ground,  airborne,  and  space  systems  are  required.  Joint  airborne  and  space  assets  are 
required  to  achieve  persistent,  global  intelligence,  surveillance,  and  reconnaissance  (ISR) 
coverage.  Joint  ground,  airborne,  and  space  assets  are  required  to  implement  the  Global 
Information  Grid  Bandwidth  Expansion  (GIG-BE)  and  are  required  for  earth  and  space 
environment  awareness.  Space  is  the  only  alternative  for  global  position,  navigation,  and 
timing  (PNT)  coverage.  The  affordable  and  timely  acquisition  of  space  systems  is  vital 
to  a  net-centric  National  Security  Strategy. 

Some  claim  that  the  process  for  acquiring  effective  military  space  systems  (and 
military  acquisitions  at  large)  is  broken,  while  others  claim  that  the  process  works 
because  US  systems  are  still  superior  to  those  of  other  nations.  DoD’s  concept  of 
maintaining  superiority  through  constant  transformation  makes  this  argument  moot  by 
recognizing  that  opportunities  for  improvement  must  be  seized  when  presented. 

Aside  from  the  important  role  that  space  systems  play  across  the  spectrum  of 
military  and  civil  operations,  space  system  acquisition  must  be  continually  reviewed  and 
maintain  a  state  of  transformation  to  overcome  several  factors  unique  to  space  system 
acquisition. 
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1.2  Background 


The  Department  of  Defense  (DoD)  decision  support  systems  guide  the  process  of 
identifying  the  need  for,  acquiring,  and  financing  materiel  solutions  when  appropriate. 
These  systems  include  the  Joint  Capabilities  Integration  &  Development  System 
(JCIDS),  the  Defense  Acquisition  System  (DAS)  or  National  Security  Space  Acquisition 
Process  (NSSAP)  for  space  systems,  and  the  Planning,  Programming,  Budgeting,  and 
Execution  (PPBE)  Process. 

Typically,  DoD  efforts  to  improve  the  overall  process  of  acquiring  a  system  have 
focused  on  only  one  of  the  decision  support  systems  at  a  time,  although  the  problems 
with  the  entire  process  may  be  attributed  to  how  the  decision  support  systems  interact, 
rather  than  the  internal  structure  of  a  single  decision  support  system.  The  most  recent 
wave  of  reform  measures  affected  the  JCIDS  and  DAS.  In  May  2003,  DoD  reissued  the 
DoD  5000  series,  including  the  DoD  Directive  5000.1,  DoD  Instruction  5000.2,  and  DoD 
Defense  Acquisition  Guidebook  with  the  major  change  being  that  evolutionary 
acquisition  was  identified  as  the  preferred  strategy  for  managing  an  acquisition  program. 
In  June  2003,  the  Chairman  of  the  Joint  Chiefs  of  Staff  (CJCS)  3170  series  was  reissued, 
replacing  the  Requirements  Generation  System  (RGS)  with  the  Joint  Capability 
Integration  and  Development  System  (JCIDS).  The  latest  refonn  measure  was  the 
creation  of  the  NSSAP  and  its  guiding  documentation,  the  National  Security  Space 
Acquisition  Policy  03-01  (NSSAP  03-01),  directing  acquisition  policy  specifically  for 
space  systems. 
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1.3  Overview 


This  thesis  looks  at  the  interaction  between  the  JCIDS  and  the  NSSAP  and 
identifies  ways  to  improve  these  processes.  The  remainder  of  this  chapter  will  present 
the  problem  the  thesis  attempts  to  address  and  provide  a  brief  description  of  the  scope, 
methodology,  limitations,  and  significance  of  this  research.  Chapter  Two  will  provide  a 
literature  review  of  current  acquisition  research,  literature,  and  process  documentation 
focusing  on  those  items  directly  related  to  the  JCIDS  and  NSSAP.  Chapter  Three  will 
describe  the  methodology  for  theory  building  from  the  qualitative  case  study.  The  case 
study  will  be  based  on  available  literature,  documentation,  and  interviews  conducted  in 
this  research.  The  results  and  recommendations  will  be  presented  and  discussed  in 
Chapter  Four.  Conclusions  and  future  research  recommendations  will  be  presented  in 
Chapter  Five. 

1.4  Problem 

This  thesis  attempts  to  answer  the  question:  what  changes  need  to  be  made  to 
enable  more  effective  interaction  between  the  National  Security  Space  Acquisition 
Process  (NSSAP)  and  the  Joint  Capabilities  Integration  &  Development  System  (JCIDS) 
to  field  affordable  space  systems? 

Ten  investigative  questions  will  be  answered  through  the  literature  review  and 
interviews  to  aid  in  developing  recommendations  to  the  above  problem.  Those  questions 
include: 

•  What  are  the  goals  of  an  EA  strategy? 

•  What  is  an  evolutionary  acquisition  (EA)  strategy? 

•  How  is  an  EA  strategy  implemented? 

•  What  development  processes  are  alternatives  to  evolutionary  acquisitions? 

•  What  are  capability-based  acquisitions? 
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•  What  are  the  goals  of  a  capability-based  strategy? 

•  How  are  capability-based  acquisitions  implemented? 

•  How  does  the  acquisition  workforce  view  the  documentation  of  JCIDS  and 

NSSAP  processes? 

1.5  Scope 

The  scope  of  research  is  within  the  capability  level  and  program  level.  The 
capability  level  includes  capabilities  that  are  delivered  in  a  hierarchical  and/or  networked 
system-of-systems  or  independent  system.  From  the  capability  level,  identifying  possible 
areas  for  improvement  within  JCIDS  requires  examination  of  an  Analysis-of- Alternatives 
(AoA)  trade  capability,  the  definition  of  capability  areas,  and  the  DoD  Architecture 
Framework  (DoDAF). 

At  the  program  level,  this  thesis  examines  the  ability  for  a  system’s  horizontal 
integration  with  other  systems,  the  integration  of  increments  with  other  increments,  and 
the  ability  of  the  acquisition  strategy  to  reflect  the  entire  scenario.  For  specific  examples, 
interviewees  were  asked  to  comment  on  any  of  three  programs.  Those  programs  were  the 
NAVSTAR  Global  Positioning  System  (GPS),  the  Space  Based  Infrared  System 
(SBIRS),  and  the  Transfonnational  Satellite  Communication  (TSAT)  System. 

1.6  Methodology 

This  thesis  uses  a  qualitative  case  study  to  frame  the  gap  between  a  capability- 
based  JCIDS  and  NSSAP  and  identify  trade  spaces  in  a  number  of  areas  which  may 
improve  the  interaction  between  the  systems.  The  literature  review  examined  research  to 
date  on  the  topic  of  project  management,  including  both  commercial  and  military 
practices,  as  well  as  available  documentation  of  the  DoD  process  and  specific  cases. 
Additional  information  was  gathered  from  field  interviews  with  various  organizations 
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involved  in  the  process.  After  analyzing  the  interview  and  documentation  data,  a  theory 
and  recommendations  are  presented  to  the  acquisition  workforce  in  this  thesis.  The  thesis 
will  be  verified  and  validated  in  defense. 

1.7  Limitations 

There  are  several  limitations  worth  stating,  although  other  limitations  surely  exist. 
The  first  is  that  only  the  interaction  between  the  JCIDS  and  the  DAS  is  examined.  The 
Planning,  Programming,  Budgeting,  and  Execution  (PPBE)  Process  is  not  examined  to 
reduce  the  complexity  of  the  thesis  and  because  the  PPBE  Process  is  largely  out  of  the 
DoD’s  control. 

Another  limitation  is  the  unique  focus  on  space  systems.  The  results  produced 
may  vary  with  non-space  programs.  Space  systems  are  excellent  candidates  to  make 
improvements  in  their  acquisition  process,  due  to  their  history  of  cost  overruns  and 
schedule  slips.  In  this  thesis  space  system  affordability  is  defines  simply  completing  a 
program  within  budget.  Also,  the  seeming  willingness  to  develop  a  system  that  addresses 
their  specific  needs  as  exemplified  by  the  adoption  of  the  NS  SAP  03-01,  and  the 
characteristics  of  their  systems  that  make  capability-based  “requirements,”  systems-of- 
systems  architectures,  and  evolutionary  strategies  so  appropriate  makes  this  research 
appropriate. 

Finally,  the  resulting  recommendations,  although  based  on  the  acquisition 
workforce  interviews  and  documentation  and  reviewed  in  defense,  will  be  untested  in 
case  study. 
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1.8  Significance 


This  work  is  highly  significant  because  of  its  timely  relevance  and  scope.  The 
question  of  how  to  improve  the  acquisition  process  is  arguably  one  of  the  most  important 
questions  facing  the  space  community,  acquisition  community,  and  DoD  at  large  -  it  is  a 
key  component  of  defense  transfonnation.  The  resulting  recommendations  may  serve  as 
an  outline  of  a  blueprint  for  modifying  the  acquisition  process  to  accommodate  the  new 
concept  of  capability-based  acquisitions. 
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II.  Literature  Review 


2.1  Introduction 

This  chapter  will  address  the  events  and  ideas  leading  to  the  development  of  the 
current  Joint  Capabilities  Integration  and  Development  System  (JCIDS)  and  National 
Security  Space  Acquisition  Process  (NSSAP)  as  described  in  the  Chairman  of  the  Joint 
Chiefs  of  Staff  (CJCSI)  Instruction/Manual  (I/M) 3 170.01  and  National  Security  Space 
Acquisition  Policy  03-01  (NSSAP  03-01).  The  chapter  will  provide  a  brief  overview  of 
the  current  decision  support  systems  that  make  up  the  Acquisition,  Technology,  and 
Logistics  process  before  highlighting  some  of  the  major  ideas  that  characterize  and 
history  of  the  JCIDS  and  NSSAP,  current  areas  of  concern  and  research,  and  official 
documentation. 

2.2  DoD  Decision  Support  Systems 

According  to  the  Defense  Acquisition  Guidebook  (DAG,  2004),  the  purpose  of 
the  Department  of  Defense  (DoD)  Decision  Support  Systems  is  that  “the  three  systems 
provide  an  integrated  approach  to  strategic  planning,  identification  of  needs  for  military 
capabilities,  systems  acquisition,  and  program  and  budget  development.”  The  three 
systems  include  the  Joint  Capabilities  Integration  and  Development  System  (JCIDS),  the 
Defense  Acquisition  System  (DAS)  for  non-space  programs  or  the  National  Security 
Space  Acquisition  Process  (NSSAP)  for  space  programs,  and  the  Planning, 
Programming,  Budgeting,  and  Execution  (PPBE)  Process.  Each  of  these  three  systems 
are  governed  by  different  documentation  and  authorities,  but  frequently  affect  the  same 
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people  and  events  throughout  the  life  time  of  an  acquisition  program,  as  illustrated  by  the 
use  of  a  Venn  Diagram  in  Figure  2.01. 


Figure  2.01  -  DoD  Decision  Support  Systems  (DAG,  2004) 


CJCS  3170.01  Series 


Joint  Capabilities 
Integration 
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DoDD  5000.1 
DoDI  5000.2 


DoD  7000.14-R 


Planning, 

Programming, 

Budgeting, 

&  Execution 
(PPBE)  Process 

DEPSECDEF 
Oversight 


As  mentioned  previously,  the  appropriate  acquisition  decision  support  systems  for 
DoD  space  systems  is  the  National  Security  Space  Acquisition  Process  which  replaces 
the  Defense  Acquisition  System,  or  blue  circle  on  the  lower  left,  in  the  figure  above. 
Also,  some  sources  refer  to  the  PPBE  Process  as  the  PPBE  System  (PPBES)  such  as  the 
OSD  Comptroller’s  website.  These  terms  will  be  used  interchangeably  throughout  this 
document.  A  correct  illustration  of  the  decision  support  systems  for  space  would  look 
something  like  the  image  in  Figure  2.02. 


Figure  2.02  -  DoD  Decision  Support  Systems  -  Space  (after  DAG,  2004) 


DoD  7000.1 4-R 


Joint  Capabilities 
Integration 
&  Development 
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In  reality,  the  center  overlap  portion  of  the  diagram  is  much  larger  and  represents 
the  area  where  all  acquisition  programs  fall.  Constant  interaction  and  coordination 
between  all  three  systems  results  in  the  most  effective  acquisition  programs  (NSSAP  03- 
01,2004). 


2.2.1  Joint  Capabilities  Integration  and  Development  System  (JCIDS) 

The  top,  yellow  circle  represents  the  Joint  Capabilities  Integration  and 
Development  System  (JCIDS).  Oversight  for  the  JCIDS  is  provided  by  the  Vice 
Chairman  of  the  Joint  Chiefs  of  Staff  and  the  Joint  Requirements  Oversight  Council  The 
CJCSI  CJCSM  3170.01  provide  guidance  for  the  JCIDS.  The  JCIDS  is  “the  systematic 
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method  established  by  the  Joint  Chiefs  of  Staff  for  assessing  gaps  in  military  joint 
warfighting  capabilities  and  recommending  solutions  to  resolve  these  gaps”  (DAG, 
2004). 

2.2.2  National  Security  Space  Acquisition  Process  (NSSAP) 

In  the  above  figures,  the  lower  left,  blue  circle  represents  the  Defense  Acquisition 
System  (DAS)  or  the  National  Security  Space  Acquisition  Process  (NSSAP).  The  DAS 
is  the  process  that  guides  acquisition  programs  (DAG,  2004)  that  meet  capability 
deficiencies  recognized  by  the  JCIDS.  The  NSSAP  does  this  for  space  systems  by  taking 
into  account  the  differences  between  space  systems  and  terrestrial  (including  air)  systems. 
Specifically,  “the  NSS  model  emphasizes  the  decision  needs  for  ‘high-tech’  small 
quantity  NSS  programs,  versus  the  DoD  5000  model  that  is  typically  focused  on  making 
the  best  large  quantity  production  decision.  The  funding  profile  for  a  typical  NSS 
program  is  usually  front-loaded  when  compared  to  a  production-focused  system.  This 
requires  the  key  decisions  for  a  NSS  program  to  be  phased  earlier  than  the  typical  DoD 
5000  milestone  decisions”  (NSSAP  03-01,  2004). 

Oversight  is  the  responsibility  of  the  Milestone  Decision  Authority  (MDA),  like 
in  the  traditional  Defense  Acquisition  System  (DAS)  for  non-space  programs.  However, 
unlike  the  DAS,  the  MDA  for  space  programs  is  the  Under  Secretary  of  the  Air  Force 
(USecAF).  The  National  Security  Space  community  includes  the  national  intelligence 
community’s  (IC)  space  programs,  in  addition  to  the  DoD’s  space  programs.  However 
the  IC  acquisition  policy  is  the  National  Reconnaissance  Office  (NRO)  Directive  82-2b, 
Acquisition  Management  -  Directive  7,  while  DoD  systems  are  governed  by  the  National 
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Security  Space  Acquisition  Policy  (again,  NSSAP)  03-01  in  place  of  the  DAS’s  DoDI 
5000.2. 

The  National  Security  Space  Acquisition  Process  for  DoD  Space  Systems  is  still 
governed  by  DoD  Directive  5000.1  in  addition  to  NSSAP  03-01.  The  relationship  with 
DoDI  5000.2  appears  to  be  less  clear.  NSSAP  03-01  (2004)  makes  mention  in  section 
3.3  of  the  space  MDA’s  right  to  waiver  or  exempt  space  programs  from  having  to  follow 
provisions  of  DoDI  5000.2  if  a  program  submits  a  memorandum  applying  for  such  a 
waiver  or  exemption.  That  excerpt  is  listed  below: 


3.3  DoDI  5000.2  Waiver  and  Exemption 
The  Space  Milestone  Decision  Authority  is  authorized  to 
approve  waivers  and  exemptions  to  provisions  of  DoD 
instructions  or  publications,  as  defined  by  DoD  Directive 
5025.1,  to  the  extent  that  the  instruction  or  publication,  and 
its  subject  matter,  are  under  the  jurisdiction  of 
USD(AT&L).  To  use  this  process,  SPD/PMs  can  request  a 
waiver  through  their  PEO  and  CAE  via  a  memo  to  the  DoD 
Space  MDA.  Once  the  DoD  Space  MDA  has  granted  the 
waiver  and  exemption,  it  remains  valid  for  the  life  of  the 
program  unless  the  DoD  Space  MDA  rescinds  the  waiver. 

(The  DoD  Space  MDA  waiver  authority  does  not  include 
DoDD  5000.1  or  other  DoD  Directives.)  For  DoD  Space 
Non-MDAPs,  the  appropriate  CAE  or  CAE-designated 
representative  (e.g.,  PEO)  has  the  authority  to  establish 
basic  acquisition  practices  and  to  act  as  the  MDA  following 
DoDI  5000.2  or  this  policy  with  approved  waiver.  (NSSAP 
03-01,2004) 

However,  a  memorandum  from  the  Under  Secretary  of  the  Air  Force  states  that 
“the  NSS  Acquisition  Policy  03-01  falls  under  the  authority  of  DoD  Directive  5000.1  and 
will  be  used  for  DoD  Space  Major  Defense  Acquisition  Programs,  replacing  processes 
and  procedures  described  in  the  DoD  Instruction  5000.2  under  the  jurisdiction  of  the 
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Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics  (USD-AT&L)” 
(USecAF,  2003). 

2.2.3  Planning,  Programming,  Budgeting,  and  Execution  (PPBE)  Process 

The  Planning,  Programming,  Budgeting,  and  Execution  (PPBE)  Process  is  “the 
Department's  strategic  planning,  program  development,  and  resource  determination 
process.  The  PPBE  process  is  used  to  craft  plans  and  programs  that  satisfy  the  demands 
of  the  National  Security  Strategy  within  resource  constraints”  (DAG,  2004).  Oversight 
for  the  PPBE  Process  is  provided  by  the  Deputy  Secretary  of  Defense.  Guidance  is 
provided  by  DoD  7000. 14-R. 

Although  the  PPBE  Process  is  one  of  the  three  DoD  decision  support  systems, 
only  the  interaction  between  the  JCIDS  and  the  NSSAP  will  be  examined  in  this  research. 
The  PPBE  Process  has  greater  external  interaction  than  the  other  two  systems  in 
determining  how  the  process  operates.  Plus,  setting  aside  the  PPBE  Process  narrows  the 
scope  of  the  research  to  something  more  manageable  given  the  time  constraint  to 
complete  the  research. 

2.3  Themes  in  Literature 

Throughout  the  course  of  literature  review,  several  themes  or  points  of  discussion 
became  evident.  These  themes  were  consistent  among  the  majority  of  reviewed  articles, 
though  not  necessarily  grouped  in  the  same  manner.  The  following  sections  provide  a 
brief  summary  of  some  of  the  most  common  themes  related  to  the  JCIDS  and  NSSAP. 
Those  themes  are: 

•  Acquisition  Reform 

•  Transformation,  Modernization,  and  Recapitalization 
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•  Capabilities  v.  Requirements 

•  Technology 

•  Evolutionary  Acquisitions 

•  Alternative  System  Development  Strategies 

2.3.1  Acquisition  Reform 

One  characteristic  of  today’s  National  Security  Space  Acquisition  Process  is  its 
emergence  from  a  20-year  reform  effort.  Table  2.01  shows  updates  to  the  three  decision 
support  systems  that  make  up  the  process  of  identifying  needs  and  providing  solutions  to 
the  warfighter.  Behind  the  updated  documents  in  the  table  are  countless  memorandums, 
directives,  codes,  instructions,  guides,  references,  architecture  frameworks,  public  laws, 
circulars,  and  orders  supporting  various  aspects  of,  or  replaced  by,  the  guiding 
documents.  Even  without  knowing  all  of  these  interim,  supporting,  or  amending 
documents,  a  pattern  in  reform  is  evident  from  the  top-level  documents  alone.  For 
example,  the  decision  support  documents  of  1991  remained  unchanged  until  1996.  The 
documents  from  1996  were  changed  by  1999  and  have  been  changed  every  year  since. 
Not  only  have  the  documents  been  reissued,  but  the  document-types  and  names  of  the 
decision  support  system  have  changed  since  1999. 

Why  the  emphasis  on  reform?  “We  have  experienced  breaches  in  nearly  every 
major  acquisition  program  and  these  breaches  are  unacceptable”  (Lord,  2006).  Breaches 
mean  violations  of  the  terms  specified  in  the  Acquisition  Program  Baseline  (APB)  of  a 
program,  pertaining  to  cost,  schedule,  and  performance.  In  other  words,  many  of  our 
systems  are  over  cost,  behind  schedule,  and  failing  to  meet  original  performance 
requirements. 

This  is  not  news  to  the  space  community.  Air  Force  Space  Command’s  (AFSPC) 
quarterly  journal  High  Frontier,  dedicated  its  entire  January  issue  to  the  subject  of  space 
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Table  2.01  -  Changes  in  the  Decision  Support  Systems  in  the  Last  Fifteen  Years 


1991 

Requirements  Generation  System 

CJCS  MOP  77 

Acquisition  Management  System 

DoD  5000  Series,  1991 

Planning,  Programming, 
Budgeting  System 

DoDD  7045.14,  1984 

1992 

Requirements  Generation  System 

CJCS  MOP  77 

Acquisition  Management  System 

DoD  5000  Series,  1991 

Planning,  Programming, 
Budgeting  System 

DoDD  7045.14,  1984 

1993 

Requirements  Generation  System 

CJCS  MOP  77 

Acquisition  Management  System 

DoD  5000  Series,  1991 

Planning,  Programming, 
Budgeting  System 

DoDD  7045.14,  1984 

1994 

Requirements  Generation  System 

CJCS  MOP  77 

Acquisition  Management  System 

DoD  5000  Series,  1991 

Planning,  Programming, 
Budgeting  System 

DoDD  7000.14,  1994 

1995 

Requirements  Generation  System 

CJCS  MOP  77 

Acquisition  Management  System 

DoD  5000  Series,  1991 

Planning,  Programming, 
Budgeting  System 

DoDD  7000.14,  1994 

1996 

Requirements  Generation  System 

CJCS  MOP  77 

Acquisition  Management  System 

DoD  5000  Series,  1996 

Planning,  Programming, 
Budgeting  System 

DoDD  7000.14,  1994 

1997 

Requirements  Generation  System 

CJCS  MOP  77 

Acquisition  Management  System 

DoD  5000  Series,  1996 

Planning,  Programming, 
Budgeting  System 

DoDD  7000.14,  1994 

1998 

Requirements  Generation  System 

CJCS  MOP  77 

Acquisition  Management  System 

DoD  5000  Series,  1996 

Planning,  Programming, 
Budgeting  System 

DoDD  7000.14,  1994 

1999 

Requirements  Generation  System 

CJCS  3170  Series,  1999 

Acquisition  Management  System 

DoD  5000  Series,  1996 

Planning,  Programming, 
Budgeting  System 

DoDD  7000.14,  1994 

2000 

Requirements  Generation  System 

CJCS  3170  Series,  1999 

Defense  Acquisition  System 

DoD  5000  Series,  2000 

Planning,  Programming, 
Budgeting  System 

DoDD  7000.14,  2000 

2001 

Requirements  Generation  System 

CJCS  3170  Series,  2001 

Defense  Acquisition  System 

DoD  5000  Series,  2000 

Planning,  Programming, 
Budgeting  System 

DoDD  7000.14,  2001 

2002 

Requirements  Generation  System 

CJCS  3170  Series,  2001 

Defense  Acquisition  System 

DoD  5000  Series,  2002 

Planning,  Programming, 
Budgeting  System 

DoDD  7000.14,  2002 

2003 

Joint  Capabilities  Integration  & 
Decision  System 

Defense  Acquisition  System 

DoD  5000  Series,  2003  &  National 

Planning,  Programming, 
Budgeting,  and  Execution 

CJCS  3170  Series,  2003 

Security  Space  Acquisition  Process 

NSSAP  03-01,  2003 

Process 

DoDD  7000.14,  2002 

2004 

Joint  Capabilities  Integration  & 
Decision  System 

CJCS  3170  Series,  £00g 

Defense  Acquisition  System 

DoD  5000  Series,  2003  &  National 
Security  Space  Acquisition  Process 

NSSAP  03-01,  2004 

Planning,  Programming, 
Budgeting,  and  Execution 
Process 

DoDD  7000.14,  2002 

2005 

Joint  Capabilities  Integration  & 
Decision  System 

CJCS  3170  Series,  2005 

Defense  Acquisition  System 

DoD  5000  Series,  2003  &  National 
Security  Space  Acquisition  Process 

NSSAP  03-01,  2004 

Planning,  Programming, 
Budgeting,  and  Execution 
Process 

DoDD  7000.14,  2002 
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acquisitions,  past,  present,  and  future.  Within  the  56  pages,  13  senior  space  professionals 
described  past  problems  and  circumstances  and  present  day  factors  that  have  led  to  the 
outcome  of  today’s  “over  cost,  behind  schedule,  and  below  performance”  reality  for 
many  space  programs.  Both  Gen  Lord  and  LtGen  Hamel  sited  events  in  the  1990s  as 
contributors  to  today’s  problems.  "Acquisition  reform  errors  of  the  1990s  left  us  with  a 
severe  lack  of  expertise  in  cost  estimating,  system  engineering,  and  program 
management"  (Lord,  2006).  And,  “in  the  early  1990s,  the  Cold  War  ended,  defense 
budgets  were  cut... the  Air  Force  systemically  cut  back  its  development  and  acquisition 
workforce  and  delayed  its  recapitalization  plans”  (Hamel,  2006).  They  also  pointed  out 
improvements  that  could  or  had  already  been  made  and  all  presented  an  impression  of 
hopefulness  for  the  future.  BrigGen  Pawlikowski  concluded  her  article  by  summarizing 
the  space  acquisition  community’s  mission.  “We  must  deliver  systems  that  are  fully 
integrated  into  effects  capability  even  if  that  means  leading  multiple  diverse  programs 
within  a  concurrent  development  environment.  This  is  our  mission  in  support  of  our 
warfighters;  failure  is  not  an  option.  The  space  acquisition  community  is  postured  to 
meet  this  mission.  We  will  overcome  the  devastating  effects  of  the  experiments  in 
acquisition  reform  in  the  1990s.” 

Many  of  the  authors  cited  a  number  of  reports  on  acquisition  reform  when 
discussing  the  basis  for  their  recommendations.  While  the  experiences  and  opinions  of 
senior  leadership  is  valuable,  these  reports  are  the  most  rigorous  research  on  the  subject 
of  acquisition  reform.  In  the  last  twenty  years,  countless  organizations  and  panels  have 
looked  at  the  issues  surrounding  acquisition  refonn,  such  as  the  1986  Blue  Ribbon 
Commission — “Packard  Commission,”  the  1990s  DSB  reports,  the  2000  Launch  Broad 
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Area  Review,  the  2001  Space  Commission  Report — “Rumsfeld  Commission,”  the  2003 
Defense  Science  Board — “Young  Panel,”  and  most  recently  the  2005  Defense 
Acquisition  Perfonnance  Assessment  (DAP A)  Project.  In  addition  to  all  of  these,  the 
Government  Accountability  Office  (GAO)  has  provided  a  number  of  valuable  reports 
pertaining  to  space  acquisition  reform. 

The  following  sections  examine  those  findings  and  recommendations  that  pertain 
specifically  to  the  JCIDS  and  NSSAP  and  were  present  in  other  articles  and  research. 
One  item  that  was  not  present  in  literature,  other  than  in  the  commission  report,  but  was  a 
frequent  topic  in  interviews  was  a  recommendation  from  the  Young  Panel.  The  Young 
Panel  recommended  that  the  space  funding  profile  should  represent  80%  for  development 
and  production  and  20%  for  operations  and  maintenance  with  a  25%  margin  for  program 
managers.  Another  interesting  characteristic  of  the  recommendations  and  problems 
identified  by  senior  leadership  and  expert  panels  is  that  at  times  the  findings  are  very 
similar  to  those  of  the  past  and  at  other  times,  the  exact  opposite  action  is  prescribed.  For 
example,  the  concept  of  Total  System  Perfonnance  Responsibility  (TSPR)  was  a  strategy 
used  to  procure  SBIRS,  based  on  1994  DAS  policy  (then  called  Acquisition  Management 
System).  LtGen  Michael  Hamel  commented  that  “the  responsibility  for  program 
management  was  largely  shifted  to,  and  shouldered  by,  the  defense  industry.  The 
construct  was  known  as  Total  System  Performance  Responsibility,  wherein  industry  was 
expected  to  deliver  end-to-end  systems.  Defense  contractors  were  given  broad  authority 
to  interpret  perfonnance  requirements,  define  system  designs,  establish  statements  of 
work  and  deliverable  items,  and  use  commercial  ‘best  practices.’”  (Hamel,  2006)  LtGen 
Hamel  continued  to  say  that  the  concept  resulted  in  lack  of  government  oversight  and 
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mission  failure.  However,  in  2003,  Dr.  Addelston  made  a  compelling  case  for  using  it  to 
allow  contractors  to  develop  end-to-end  system-of-systems  based  on  a  government 
CONOPS  (Addelston,  2003).  This  fluctuation  may  reflect  the  difficulty  in  developing  a 
comprehensive  system  that  is  still  flexible  or  it  may  indicate  that  something  else  is  wrong 
other  than  the  system. 


2.3.2  Transformation,  Modernization,  and  Recapitalization 

Another  theme  prevalent  in  many  articles  focusing  on  all  processes  associated 

with  DoD  acquisitions,  is  the  concept  of  “Transformation.”  Transformation  simply 

means  to  change  or  modernize  many  aspects  of  the  DoD  to  be  more  flexible  to  threats 

(Myers,  2001).  The  Defenselink  Transfonnation  website  provides  the  following 

overview  to  describe  transformation: 

The  Sept.  11,  2001,  terror  attacks  on  the  United  States 
accelerated  the  need  to  transform  to  better  meet  the 
challenges  of  the  21st  century,  thus,  sustain  American 
competitive  advantage  in  warfare. 

Transformation  is  foremost  a  continuing  process  that  does 
not  have  an  end  point.  It  is  meant  to  create  or  anticipate  the 
future.  Transformation  is  meant  to  deal  with  the  co¬ 
evolution  of  concepts,  processes,  organizations  and 
technology.  Change  in  any  one  of  these  areas  necessitates 
change  in  all. 

Transformation  is  meant  to  create  new  competitive  areas 
and  new  competencies.  It  is  meant  to  identify,  leverage  and 
even  create  new  underlying  principles  for  the  way  things 
are  done.  Transfonnation  is  meant  to  identify  and  leverage 
new  sources  of  power.  The  overall  objective  of  these 
changes  is  simply  -  sustained  American  competitive 
advantage  in  warfare.  (Defenselink  Transformation 
Website,  2006) 
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Transformation  was  not  born  from  the  9-11  terror  attacks.  It  was  included  in  the 


Space  Commission’s  January  2001  Report.  The  Space  Commission  defined  it  similarly 
as  to  “develop,  deploy,  and  maintain  the  means  to  deter  attack  on  and  to  defend 
vulnerable  space  capabilities... this  requires  a  deterrence  strategy  for  space,  which  in  turn 
must  be  supported  by  a  broader  range  of  space  capabilities”  (Space  Commission,  2001). 
The  Space  Commission  defined  these  capabilities  necessary  to  space  transformation  as: 
Deterrence  and  Defense,  Policy  for  Space  Assured  Access  to  Space  and  On-Orbit 
Operations,  Space  Situational  Awareness,  Earth  Surveillance  from  Space,  Global 
Command,  Control  and  Communications  in  Space,  Defense  in  Space,  Homeland 
Defense,  and  Power  Projection  In,  From,  and  Through  Space  (Space  Commission, 
2001). 

In  fact,  the  concept  of  transfonnation,  or  modernization,  has  been  around  for 
some  time.  A  Feb  1995  DoD  news  release  on  the  FY  1996-97  Defense  Budget  quoted 
Secretary  Perry  saying  that  the  budget  would  be  used  in  part  for  “prudent  weapons 
modernization”  (DoD,  1995).  The  article  later  substituted  recapitalization  for 
modernization  when  it  said  that  “The  new  budget  plans  also  begin  the  ‘recapitalization’ 
of  U.S.  forces— that  is,  the  modernization  of  weapons  and  equipment,  after  several  years 
in  which  the  Department  lived  off  its  Cold  War  stocks  of  equipment”  (DoD,  1995).  And 
even  as  recently  as  June  of  2005,  while  testifying  before  the  Senate  Anned  Services 
Committee  (SASC)  during  his  confirmation  hearing,  Gen  Moseley  stated  that 
recapitalization  of  the  aging  aircraft  fleet  was  his  third  priority  (third  to  improving  joint 
warfighting  and  strengthening  Air  Force  people)  (Gettle,  2005). 
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The  concepts  of  modernizing,  recapitalization,  and  transfonnation  as  synonyms  is 
not  new.  However,  today’s  usage  implies  that  recapitalization  specifically  applies  to 
modernizing  weapon  systems  after  a  period  of  allowing  systems  to  remain  at  the  status 
quo  and  fall  behind  state  of  the  art.  Transformation  has  also  become  more  clearly 
defined.  In  2003,  the  Secretary  of  Defense  published  the  Transfonnation  Planning 
Guidance.  Within  the  document,  Secretary  Rumsfeld  stated  that  transformation  is  more 
than  modernizing  weapon  systems  in  that  it  includes  continually  transfonning  our  people, 
processes,  and  military  forces  “so  that  our  armed  forces  are  always  several  steps  ahead  of 
any  potential  adversary”  and  can  react  quickly  to  whatever  the  threat  may  be  including 
terrorist  attacks,  cyber-war  attacks,  traditional  state-on-state  attacks,  etc.  (DoD  TPG, 
2003). 

The  document  also  specifies  that  transfonnation  efforts  will  occur  in  three  areas: 
how  we  fight,  how  we  do  business,  and  how  we  work  with  others  (DoD  TPG,  2003). 
Transforming  how  we  fight  means  supporting  the  four  pillars  of  transformation  identified 
in  the  2001  QDR  and  pictured  in  Figure  2.03. 

Figure  2.03  -  2001  QDR  Transformation  Pillars  (DoD  TPG,  2003) 
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The  Office  of  Force  Transformation  has  established  the  following  five  goals,  which 
closely  mirror  those  of  the  2001  QDR  and  support  the  pillars.  Those  goals  are  listed 
below: 

1.  Make  force  transformation  a  pivotal  element  of  national 
defense  strategy  and  DoD  corporate  strategy  effectively 
supporting  the  four  strategic  pillars  of  the  national  military 
strategy. 

2.  Change  the  force  and  its  culture  from  the  bottom  up 
through  the  use  of  experimentation,  transformational 
articles  (operational  prototyping)  and  the  creation  and 
sharing  of  new  knowledge  and  experiences. 

3.  Implement  Network  Centric  Warfare  (NCW)  as  the  theory 
of  war  for  the  information  age  and  the  organizing  principle 
for  national  military  planning  and  joint  concepts, 
capabilities  and  systems. 
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4.  Get  the  decision  rules  and  metrics  right  and  cause  them  to 
be  applied  enterprise  wide. 

5.  Discover,  create  or  cause  to  be  created  new  military 
capabilities  to  broaden  the  capabilities  base  and  mitigate 
risk.  (Office  of  Force  Transformation  Website,  2005) 

Transforming  how  we  do  business  includes  a  number  of  efforts;  most  notable  is 
“adaptive  planning,  a  more  entrepreneurial,  future-oriented  capabilities-based  resource 
allocation  planning  process,  accelerated  acquisition  cycles  built  on  spiral  development, 
output-based  management,  and  a  reformed  analytic  support  agenda”  (DoD  TPG,  2003). 
Furthermore,  reforming  the  acquisition  process  is  a  top  priority  (DoD  TPG,  2003).  “The 
Department  [of  Defense]  is  reducing  acquisition  cycle  time  and  aligning  acquisition  with 
a  new  capabilities-based  resource  allocation  process  built  around  joint  operating 
concepts.  Instead  of  building  plans,  operations  and  doctrine  around  individual  military 
systems  as  often  occurred  in  the  past,  henceforth  the  Department  will  explicitly  link 
acquisition  strategy  to  future  joint  concepts  in  order  to  provide  the  capabilities  necessary 
to  execute  future  operations”  (DoD  TPG,  2003). 

Acquisition  reform,  whether  a  new  concept  or  not,  was  clearly  needed  as 
demonstrated  by  the  multitude  of  GAO,  DSB,  and  other  special  panels  chartered  to  look 
at  the  Acquisition  process  discussed  in  section  2.2.1.  That  need  was  brought  to  the 
forefront  after  the  events  of  9- 1 1  demonstrated  interagency  communication  weaknesses 
and  drastically  different  threats  than  DoD  had  prepared  for  through  traditional  methods  of 
adjusting  Doctrine,  Organization,  Training,  Materiel,  Leadership,  Personnel,  and 
Facilities  (DOTMLPF).  Therefore,  the  concept  of  transformation,  though  introduced 
before  9-11,  gained  substantial  support  because  it  represented  more  than  system 
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modernization  or  recapitalization.  It  also  emphasized  joint  activities,  the  development  of 
a  net-centric  environment,  and  promoting  changes  across  the  DOTMLPF  spectrum. 

2.3.3  Capabilities  v.  Requirements 

In  May  of  2003,  the  Joint  Capabilities  Integration  &  Development  System 
(JCIDS)  replaced  the  Requirements  Generation  System  (RGS)  decision  support  system. 
According  to  a  briefing  given  by  the  Joint  Staff,  the  former  RGS  had  a  number  of 
shortcomings  including  producing  stove-piped  systems  that  were  not  integrated  with 
other  systems;  producing  service  oriented  requirements,  rather  than  joint  focused;  lacking 
an  objective  construct  for  proposal  analyses;  duplication  of  efforts  in  smaller  programs; 
and  unprioritized  joint  warfighting  needs.  (Joint  Staff  /  J8,  2005)  Figure  2.04  compares 
the  fonner  RGS  with  the  new  JCIDS. 


Figure  2.04  -  RGS  v.  JCIDS  Comparison  (Joint  Staff  /  J8,  2005) 
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In  addition  to  the  top-down,  joint-approach,  an  additional  JCIDS  characteristic  is 
the  prevalent  use  of  the  term  “capability”  and  the  reduced  use  of  the  term  “requirements.” 
This  has  led  to  discussion  on  what  exactly  is  meant  by  capability  and  related  terms  such 
as  capability-based  acquisitions.  The  governing  document  for  the  JCIDS  process,  CJCSI 
3170  defines  a  capability  [as]  “the  ability  to  achieve  a  desired  effect  under  specified 
standards  and  conditions  through  combinations  of  means  and  ways  to  perfonn  a  set  of 
tasks”  (CJCSI  3170,  2005).  Air  Force  Instruction  10-601,  Capabilities  Based 
Requirements  Development,  defines  capability  as  “the  ability  to  execute  a  specified 
course  of  action.  It  is  defined  by  an  operational  user  and  expressed  in  broad  operational 
terms  in  the  format  of  an  initial  capabilities  document  (ICD)  or  a  DOTMLPF  change 
recommendation.  In  the  case  of  materiel  proposals,  the  definition  will  progressively 
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evolve  to  DOTMLPF  performance  attributes  identified  in  the  [capability  development 
document]  CDD  and  the  [capability  production  document]  CPD”  (AFI  10-601,  2004). 
The  DoD  Dictionary  defines  capability  as  “the  ability  to  execute  a  specified  course  of 
action.  (A  capability  may  or  may  not  be  accompanied  by  an  intention.)”  AFI  63-101, 
Operations  of  Capability  Based  Acquisition  System  and  AFPD  63-1,  Capability-Based 
Acquisition  System  do  not  define  or  refer  to  definitions  of  a  capability. 

The  failure  to  adopt  joint  capability-based  acquisitions  in  implementation  and 
across  services  was  noted  in  a  recent  report  by  the  Government  Accountability  Office, 
titled  “DoD  Management  Approach  and  Process  Not  Well-Suited  to  Support 
Development  of  Global  Information  Grid  (GIG).”  In  the  report,  the  GAO  stated  that 
“DoD  program  management  and  acquisition  oversight  tend  to  focus  on  individual 
programs  and  not  necessarily  on  synchronizing  multiple  programs  to  deliver 
interdependent  systems  at  the  same  time,  as  required  to  achieve  the  intended  capability” 
(GAO,  2006). 

An  additional  characteristic  of  the  JCIDS  is  the  option  of  using  an  architecture 

made  of  family-of-systems  (FoS)  or  system-of-systems  (SoS)  as  solutions.  CJCSI 

identifies  FoS  and  SoS  as  follows: 

Family  of  Systems  -  A  set  of  systems  that  provide  similar 
capabilities  through  different  approaches  to  achieve  similar 
or  complementary  effects.  For  instance,  the  warfighter  may 
need  the  capability  to  track  moving  targets.  The  FoS  that 
provides  this  capability  could  include  unmanned  or  manned 
aerial  vehicles  with  appropriate  sensors,  a  space-based 
sensor  platform  or  a  special  operations  capability.  Each  can 
provide  the  ability  to  track  moving  targets,  but  with 
differing  characteristics  of  persistence,  accuracy, 
timeliness,  etc. 
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System  of  Systems  -  A  set  or  arrangement  of 
interdependent  systems  that  are  related  or  connected  to 
provide  a  given  capability.  The  loss  of  any  part  of  the 
system  will  significantly  degrade  the  performance  or 
capabilities  of  the  whole.  The  development  of  a  SoS 
solution  will  involve  trade  space  between  the  systems  as 
well  as  within  an  individual  system  performance.  An 
example  of  a  SoS  would  be  a  combat  aircraft.  While  the 
aircraft  may  be  developed  as  a  single  system,  it  could 
incorporate  subsystems  developed  for  other  aircraft.  For 
example,  the  radar  from  an  existing  aircraft  may  be 
incorporated  into  the  one  being  developed  rather  than 
developing  a  new  radar.  The  SoS  in  this  case  would  be  the 
airframe,  engines,  radar,  avionics,  etc.  that  make  up  the 
entire  combat  aircraft  capability.  (CJCSI  3170.01,  2005) 

The  concept  of  joint,  net-centric,  SoS  or  FoS  architectures  has  caused  a  number  of 
organizations  to  take  a  look  at  several  areas  within  DoD  acquisitions  aside  from  those 
associated  with  the  oversight  and  review  processes  detailed  in  CJCSI  3170  and  NS  SAP 
03-01.  In  February  of  2004,  the  National  Defense  Industrial  Association  (NDIA) 
Systems  Engineering  Division  Modeling  &  Simulation  (M&S)  Community  published 
their  findings  on  M&S  support  and  how  it  related  to  the  new  DoD  5000  series.  The 
report  had: 

•  13  findings  on  the  systems  engineering  process 

•  35  findings  regarding  M&S 

•  four  recommendations  on  the  use  of  M&S 

•  12  recommendations  on  enabling  the  use  of  M&S  (NDIA,  2004) 

In  April  of  2005,  the  Aerospace  Corporation  published  a  draft  rewrite  of  Military 
Standard  499C,  Systems  Engineering — its  first  update  since  1974.  The  document  is  one 
example  of  LtGen  Hamel’s  “comprehensive  fixes”  at  Space  and  Missile  Systems  Center 
(SMC).  “These  comprehensive  fixes  include  reestablishing  systems  engineering 
discipline,  critical  development  processes,  tailored  military  specifications  and 
standards...”  (Hamel,  2006).  In  addition  to  the  implications  of  capability-based 
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acquisitions  on  SE  and  M&S,  testing  and  evaluation  (T&E)  is  also  under  review.  Col 
Eileen  Bjorkman  is  the  Joint  Test  &  Evaluation  Methodology  (JTEM)  Feasibility  Study 
Director  charged  with  “developing  methods  and  processes  for  testing  in  a  Joint 
environment”  (Bjorkman,  2005).  As  is  the  case  with  many  Joint  Acquisition  activities, 
Bjorkman  says  that  relationships  need  to  be  established  among  competing  processes  and 
organizations  (Bjorkman,  2005). 

2.3.4  Technology 

The  role  of  technology  in  the  acquisition  processes  was  another  strong  theme  for 
discussion  in  the  literature.  These  discussions  included  the  military  lag  behind  the 
commercial  world  in  technological  advancements,  DoD’s  lag  behind  national  space 
systems,  and  incorporating  immature  technology  into  DoD  systems. 

A  number  of  articles  and  reports  highlighted  a  need  “to  give  DOD  access  to  those 
technologies,  products,  and  processes  which  are  dominated  by  the  commercial  market 
place.  Electronics,  software,  computer  systems,  telecommunications,  and  flexible 
manufacturing  are  example  areas  where  commercial  technology  is  far  more  advanced 
than  military  technology”  (DSB  1993). 

Some  articles  stated  that  there  was  too  much  technology  development  during  the 
acquisition  life  cycle.  In  a  speech  given  to  a  graduating  class  of  the  Defense  Systems 
Management  Course,  Mr.  John  Wilson  said  that  the  Acquisition  community  needed  to 
consciously  separate  technology  development  from  acquisition  (Johnson,  1999). 

The  1993  DSB  sited  one  problem  with  the  acquisition  process  as  “assuming  ‘this 
will  be  the  last  new  system  for  the  next  two  decades,’  and  including  all  new  (often 
unproven)  technology  at  the  start  of  full-scale  development,  and  adding  new  requirements 
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to  this  same  system  over  time”  (DSB,  1993).  Similarly,  Mr.  Wilson  said  “Because  we 
expect  each  generation  of  technology  to  be  a  revolutionary  leap  ahead  of  the  last 
generation,  we  try  to  fund  requirements  ten  to  fifteen  years  in  the  future.  As  the  F-22 
example  demonstrates,  not  only  does  this  practice  cause  us  to  design  systems  based  on 
our  best  guess  of  future  threats  and  technology,  which  is  often  inaccurate,  but  it  also 
extends  cycle  times  by  making  us  repeatedly  revise  the  program  to  incorporate  new 
developments”  (Johnson,  1999).  This  problem  begs  the  question:  should  defense  system 
technology  be  revolutionary  or  evolutionary? 

According  to  the  1999  DSB,  “there  have  been  few  revolutionary  programs  in  the 
past,  that  have  succeeded  without  the  strong  support  of  the  owning  Service.  Major 
change  almost  always  involves  some  leadership  group  who  perceives  a  pending  crisis.  If 
the  DoD  wants  to  make  change,  it  must  recognize  the  difficulty  of  sustaining  funding 
support  for  revolutionary  changes,  and  then  provide  the  leadership  for  giving  such 
programs  funding  stability”  (DSB  1999).  Gen  Hamel,  among  others  (Sugar,  2006) 
(Stevens,  2006),  feel  that  “program  stability  is  essential  if  we  are  to  avoid  continuous  re¬ 
planning  and  re -baselining,  which  inevitably  causes  delays  and  cost  growth”  (Hamel, 
2006). 

2.3.5  Evolutionary  Acquisitions  (EA) 

Many  believe  that  an  evolutionary  strategy,  as  opposed  to  a  single  step  strategy 
providing  revolutionary  change,  will  solve  Acquisition’s  problem  of  lengthy  acquisition 
cycle  time.  The  Packard  Commission  stated  that  “an  unreasonable  long  acquisition  cycle 
of  10  to  15  years  for  major  weapons  systems  is  a  central  problem  from  which  most  other 
acquisition  problems  stem”  (Packard,  1986)  including  obsolete  technology,  cost  growth, 
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and  “gold  plated”  defense  systems  (Packard,  1986).  Cycle  time  is  the  length  of  time  it 
takes  to  go  from  program  initiation  to  initial  operational  capability  (IOC)  (Johnson, 
1999).  Figure  2.05  depicts  the  concept  of  acquisition  cycle  time  according  to  the  1999 
DSB. 


Figure  2.05  -  Acquisition  Cycle  Time  (DSB,  1999) 


“Rather  than  aiming  for  a  100%  solution  that  takes  20  years  to  complete,  they  will  strive 
for  an  80%  solution  that  can  be  fielded  to  the  troops  faster  and  at  affordable  prices.” 
(Farrell,  2002)  DoD  supported  this  notion  by  stating  that  Evolutionary  Acquisition  (EA) 
was  the  preferred  method  for  acquiring  weapon  systems  in  the  2003  reissuance  of  the 
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DoD  5000  series.  The  NS  SAP  03-01  followed  suit,  also  stating  that  EA  was  the 
preferred  method  for  developing  space  systems. 

What  is  EA?  According  to  DoDI  5000.2, 

An  evolutionary  approach  delivers  capability  in  increments, 
recognizing,  up  front,  the  need  for  future  capability 
improvements.  The  objective  is  to  balance  needs  and 
available  capability  with  resources,  and  to  put  capability 
into  the  hands  of  the  user  quickly.  The  success  of  the 
strategy  depends  on  consistent  and  continuous  definition  of 
requirements,  and  the  maturation  of  technologies  that  lead 
to  disciplined  development  and  production  of  systems  that 
provide  increasing  capability  towards  a  materiel  concept.... 

The  approaches  to  achieve  evolutionary  acquisition  require 
collaboration  between  the  user,  tester,  and  developer.  They 
include: 

Spiral  Development  -  In  this  process,  a  desired  capability  is 
identified,  but  the  end-state  requirements  are  not  known  at 
program  initiation.  Those  requirements  are  refined  through 
demonstration  and  risk  management;  there  is  continuous 
user  feedback;  and  each  increment  provides  the  user  the 
best  possible  capability.  The  requirements  for  future 
increments  depend  on  feedback  from  users  and  technology 
maturation. 

Incremental  Development  -  In  this  process,  a  desired 
capability  is  identified,  an  end-state  requirement  is  known, 
and  that  requirement  is  met  over  time.  (DoDI  5000.2, 
2003) 


NSSAP  03-01  defines  EA  similarly  and  goes  on  to  say  that  “evolutionary 
acquisition  has  been  a  cornerstone  for  space  system  development  since  the  early  1960’s” 
(NSSAP,  2004)  even  though  the  recognition  in  DoD  has  been  fairly  recent  and  can  be 
traced  back  to  DoD  computer  system  acquisition  (Farkas,  2003). 

Traces  of  an  evolutionary  concept  can  be  found  dating  back  to  the  1930s  when 
Walter  Shewhart  of  Bell  Labs  proposed  a  series  of  small  “Plan-Do-Study-Acf  ’  cycles,  a 
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concept  referred  to  as  PDSA,  to  improve  the  quality  of  products.  (Larman,  2003)  W. 
Edwards  Deming,  also  a  quality  expert,  promoted  PDSA  in  the  1940s  and  included  it  in 
his  1986  book  Out  of  the  Crisis  (Deming,  1986).  In  the  1950s,  the  X-15  hypersonic  jet 
incorporated  an  iterative  and  incremental  development,  or  IID,  strategy.  NASA  credited 
some  of  the  success  of  the  project  to  the  IID  strategy.  The  IID  strategy  was  also  used  on 
NASA’s  software  for  the  Project  Mercury  program  (Larman,  2003).  It  was  in  the  mid- 
1950s  that  IID  became  widely  accepted  in  the  software  programming  community.  In 
1976,  Tom  Gilb  introduced  the  concept  of  evolutionary  project  management  and  in  the 
mid-1980s  TRW’s  Barry  Boehm  coined  the  term  “Spiral  Development”  (Larman,  2003). 
Figure  2.06  illustrates  Boehm’s  concept  of  spiral  development. 

In  the  spiral  model,  the  desired  capability  is  achieved  at  the  end  of  the  entire 
spiral.  Boehm’s  model  releases  prototypes  for  testing,  but  does  not  consider  them  to  be 
solutions.  Some  consider  the  spiral  model  to  be  a  system  production  model  (Unger, 
2003)  but  apply  different  steps  than  the  one  developed  by  Boehm.  Unger  describes 
product  development  models  that  use  time  or  budget  constraints  to  mark  the  end  of  a 
development  phase.  The  automobile  industry  for  example,  makes  a  number  of  upgrades 
or  modifications  in  about  18  months,  fast  enough  to  have  a  new  edition  of  the  Ford 
Explorer  available  each  year.  This  is  an  example  of  time-block  product  development  and 
an  evolutionary  strategy,  but  it  is  made  possible  through  a  fairly  mature  prototype  that 
remains  constant.  Software  engineering  is  another  industry  that  releases  mature 
prototypes  without  requiring  too  many  spirals. 

Figure  2.06  -  Spiral  Development  (Boehm,  2001) 
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The  National  Security  Space  Acquisition  Process’  application  treats  prototypes  as 
militarily  useful,  improved  capability  systems  that  may  or  may  not  be  brought  to  the 
same  level  as  the  final  capability,  when  the  spiral  is  completed.  The  ability  to  bring  an 
earlier  system  to  the  same  level  as  a  new  system  presents  problems  for  space  systems 
since  only  software  can  be  upgraded  on  a  launched  satellite  at  this  time.  Capability  is 
generally  added  in  following  generations  of  satellites  by  designing  them  to  be  backward 
compatible  with  earlier  generations. 

This  may  seem  similar  to  block  development  or  preplanned  product  improvement 
(P3I)  and  in  some  ways,  it  is;  which  is  why  the  space  community  says  that  they  have  been 
using  EA  strategies  for  years  (NSSAP  03-01,  2004).  Pete  Aldridge  published  the 
following  definitions: 
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Increment  or  Block  -  A  militarily  useful  and  supportable 
operational  capability  that  can  be  effectively  developed, 
produced  or  acquired,  deployed,  and  sustained.  Each 
increment  of  capability  will  have  its  own  set  of  thresholds 
and  objectives  set  by  the  user. 

Preplanned  Product  Improvement  -  A  traditional 
acquisition  strategy  that  provides  for  adding  improved 
capability  to  a  mature  system.  (Aldridge,  2002) 

In  DoD’s  application  of  EA,  EA  (spiral  or  incremental),  block  development,  and  P3I 

ALL  result  in  blocks  or  increments.  The  difference  is  in  the  expected  capability.  EA 

shoots  for  an  80%  solution  upgraded  over  time,  block  shoots  for  a  100%  solution  with 

each  successive  block,  and  P3I  shoots  for  a  100%  solution  with  planned  upgrades  to 

follow  (Farkas  &  Farmer,  2005).  Figure  2.07  is  from  Crosstalk  Magazine  and  attempts  to 

capture  these  concepts  (Crosstalk,  2002). 


Figure  2.07  -  EA  Visualization  (Crosstalk,  2002) 


Increment  or 

Block  1 

♦ 

v/V^ 

Spirals 

lnci?m?nt  or  Evolutionary 

Block  2  .  .  ...  7 

^  Acquisition 

v/X/' 

Spirals 

Increment  or 

Block  3 

♦ 

v/V^ 

Spirals 

Increment  or 

Block  4 

♦ 

v/V^ 

Spirals 

32 


Mr.  David  Brown,  of  the  Technology  and  Engineering  Department  in  the  Defense 
Acquisition  University  (DAU),  briefed  the  following  figure  at  the  2005  Space  Systems 
Engineering  and  Acquisition  Excellence  Forum. 


Figure  2.08  -  Linear  Sequential  Acquisition  Process  v.  Evolutionary  Acquisition  Process 
(Brown,  2005) 
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2.3. 6  System  Development  Strategies 

To  software  and  systems  engineers,  as  well  as  civilian  project  managers, 
traditional,  spiral,  and  incremental  development  are  defined  and  illustrated  differently 
than  the  DoD  definitions.  To  the  DoD,  the  use  of  spiral  or  incremental  development  is 
determined  by  knowledge  of  an  end  state.  DoD  deploys  improved  usable  systems  at  the 
end  of  a  development  phase.  Adding  to  some  of  the  confusion,  aside  from  definitions 
differing  from  the  systems  engineering  community  or  commercial  practices,  is  the 
inconsistency  in  images  to  portray  DoD’s  definitions.  The  following  sections  address 
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various  development  strategies  used  commercially,  grounded  in  systems  engineering, 
software  development,  and  project  management  practices. 

The  Project  Management  Institute  summarizes  a  number  of  development  methods 
in  a  paper  titled  “Software  Development  and  Linearity.”  The  paper  describes  “generally 
accepted”  (Wideman,  2003)  program  management  practices  in  software  development, 
which  is  appropriate  for  all  DoD  space  systems  because  of  their  software  complexity  and 
the  push  towards  interoperability  in  a  joint  community  through  netcentricity. 

The  first  model  discussed  is  the  waterfall  model,  picture  in  Figure  2.09;  the 
waterfall  model  is  also  pictured  in  Figure  2.09  as  the  linear-sequential  acquisition 
process.  Wideman  describes  “good”  and  “not  so  good”  features  of  the  waterfall  model. 


Figure  2.09  -  The  Waterfall  Model  (Buede,  2000) 
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Good  Features: 

•  The  waterfall  approach  has  been  around  for  a  long  time,  and  many  people 
are  familiar  and  comfortable  with  it. 

•  It  is  simple  and  easily  understood. 

•  It  does  recognize  the  need  to  move  one  stage  at  a  time  and  recycle  back  to 
the  previous  stage  to  validate  the  stage  outcome.  (Wideman,  2003) 

Not  So  Good  Features: 

•  The  waterfall  model  approach  does  not  satisfy  the  requirement  for 
executive  control. 

•  It  is  very  difficult  to  manage  under  conditions  of  complexity. 

•  In  the  waterfall  approach,  integration  and  testing  is  generally  left  until  the 
end.  That’s  when  “all  the  chickens  come  home  to  roost,”  with  disastrous 
effect  on  project  cost  and  schedule.  (Wideman,  2003) 

The  second  model  discussed  is  the  “Vee  Model”  pictured  in  Figure  2.10.  The 
National  Reconnaissance  Office  (NRO)  uses  a  similar  model.  That  model  is  pictured  in 
Figure  2.1 1  with  an  overlay  showing  when  key  reviews  take  place. 

Good  Features: 

•  Most  people  who  have  an  engineering  background  are  very  comfortable 
with  the  systems  approach. 

•  It  is  very  good  wherever  it  is  possible  to  describe,  i.e.  specify,  the 
requirements  with  high  degree  of  certainty 

•  The  acquiring  authority  requires  a  thoroughly  well-documented  track 
record  or  audit  trail 

•  Consequently,  it  is  popular  with  big  government  departments, . . . 

•  Where  money  is  not  the  limiting  criteria,  though  competitive  bidding 
might  be.  (Wideman,  2003) 

Not  So  Good  Features: 

•  The  process  is  heavy  on  documentation. 

•  It  assumes  that  it  is  possible  to  arrive  at  near-perfect  documentation  that  is 
complete,  and  is  truly  representative  of  the  ultimate  “requirements,”. . . 

•  And  can  be  frozen,  and  the  authors,  i.e.  the  stakeholders,  can  be  held 
accountable  to  those  requirement  specifications.  (Wideman,  2003) 
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Figure  2.10  -  The  Vee  Model  (Buede,  2000) 
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Figure  2.11-  The  NRO  Vee  Model  (NRO,  2000) 
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SDR  -  System  Design  Review 
DCR  -  Design  Concept  Review 
PDR  -  Preliminary  Design  Review 
CDR  -  Critical  Design  Review 
SCR  -  Segment  Completion  Review 
TRR  -  Test  Readiness  Review 
PSR  -  Pre-Ship  Review 
PLR  -  Pre  Launch  Review 
XRR  -  Transition  Readiness  Review 
ORR  -  Operational  Readiness  Review 
OAR  -  Operational  Acceptance  Review 


Atrr 


BUILD 

■Fabricate  HW 
Assembly 
■Develop  SW 
Components 


Time  and  Maturity 
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The  third  model  discussed  is  the  spiral  model.  Wideman  points  out  that  spiral 

development  means  different  things  to  different  people  (Wideman,  2003)  but  takes  the 

most  classical  view  in  his  interpretation  that  it  consists  of  four  management  processes 

with  four  cycles  through  those  processes  (Wideman,  2003).  The  four  phases  are: 

identify,  design,  construct,  and  evaluate.  The  four  cycles  are: 

Proof-of-concept  cycle  —  define  the  business  goals, 
capture  the  requirements,  develop  a  conceptual  design, 
construct  a  "proof-of-concept",  establish  test  plans,  conduct 
a  risk  analysis.  Share  results  with  user. 

First-build  cycle  —  derive  system  requirements,  develop 
logic  design,  construct  first  build,  evaluate  results.  Share 
results  with  user. 

Second-build  cycle  —  derive  subsystem  requirements, 
produce  physical  design,  construct  second  build,  evaluate 
results.  Share  results  with  user. 

Final-build  cycle  —  derive  unit  requirements,  produce  final 
design,  construct  final  build,  test  all  levels.  Seek  user 
acceptance.  (Wideman,  2003) 

Good  Features: 

•  In  this  approach,  the  entire  application  is  built  working  with  the  user. 

•  Any  gaps  in  requirements  are  identified  as  work  progresses  into  more 
detail. 

•  The  process  is  continued  until  the  code  is  finally  accepted. 

•  The  diagrammatic  representation,  i.e.  the  spiral,  does  convey  very  clearly 
the  cyclic  nature  of  the  process. 

•  And  it  also  conveys  the  progression  through  the  project  life  plan. 
(Wideman,  2003) 

Not  So  Good  Features: 

•  This  approach  requires  serious  discipline  on  the  part  of  the  users. 

•  If  the  users  are  not  responsible  for  the  schedule  and  budget,  as  very  often 
they  are  not,  executive  control  can  be  difficult. 

•  For  a  software  developer  working  under  a  firm-price  contract,  it  may  be 
impossible. 

•  The  model  depicts  four  cycles.  However,  if  cycles  are  added  indefinitely 
for  “just  one  more  tweak”  then  eventually... Everyone  gives  up  in 
frustration! 
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•  Or,  the  time  and  money  runs  out.(Wideman,  2003) 

Like  Wideman,  Boehm  and  Brown  both  warn  against  incorporating  too  many 
spirals,  or  creating  a  program  management  “death  spiral”  (Brown,  2004)  resulting  in 
running  out  of  patience,  time,  and  money;  however,  DoD’s  definition  of  spiral 
development  makes  no  stipulation  as  to  the  end  date  for  a  capability  or  system 
development. 

Buede  discusses  a  fourth  development  model  that  seems  to  more  closely  resemble 
the  chart  published  by  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  (USD(AT&L))  illustrating  the  interaction  between  the  three  decision  support 
systems.  This  model  describes  incremental  development  using  the  Vee  Model  and  is 
pictured  below  in  Figure  2.12.  Under  this  model,  a  useful  subset  of  the  system  is 
produced  initially  followed  by  upgrades  and  system  expansions  until  the  entire  system  is 
operational.  (Buede,  2000)  This  definition  seems  most  similar  to  the  DoD  definition  for 
Spiral  and  Incremental  Development,  without  referring  to  the  user’s  role  or  knowledge  of 
the  desired  end  state.  Buede  also  points  out  that  while  some  people  believe  that  one 
needs  to  choose  between  the  traditional  Vee  or  the  Spiral  model,  research  has  shown 
“that  the  spiral  activities  can  be  mapped  onto  the  Vee  model  without  swapping  any 
activities  in  time.”  Both  the  Iterative  Vee  and  the  Spiral-mapped  Vee  are  pictured  below 
in  Figures  2.12  and  2.13. 
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Figure  2.12  -  The  Iterative  Vee  (Buede,  2000) 


Time 


Figure  2.13  -  The  Spiral-mapped  Vee  (after  Buede, 

2000) 


Proof-of-Concept 

Cycle 


First-build 

Cycle 


Second-build 

Cycle 


Final-build 

Cycle 
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2.4  Summary 

This  chapter  introduced  the  DoD  decision  support  system,  with  an  emphasis  on 
the  JCIDS  and  NSSAP.  Documentation,  research,  and  articles  pertaining  to  these 
decision  support  systems  was  reviewed  and  presented  by  major  theme,  for  the  purpose  of 
answering  the  investigative  questions  listed  in  Chapter  One  and  shaping  the  additional 
research  described  in  Chapter  Three. 

Acquisition  challenges  have  existed  since  the  days  of  George  Washington  (Lord, 
2006).  But  the  string  of  modifications,  some  minor  and  many  major,  indicate  that 
acquisition  reform  is  a  real  issue  because  an  acceptable  policy  has  not  been  developed  or 
has  not  been  implemented  properly. 

The  2001  QDR  and  2003  Transformation  Planning  Guidance  identify  very 
clearly,  a  number  of  objectives  for  DoD  transformation — acquisitions  included.  Those 
objectives  are  further  highlighted  in  the  JCIDS  and  NSSAP  guiding  documentation  and 
can  be  summarized  as  joint  operations,  intelligence  exploitation  (through  netcentricity), 
developing  new  warfighting  concepts  (capability-based),  and  delivering  new  capability 
(evolutionary).  However,  supporting  documentation  specifying  implementation  of  these 
concepts  appears  to  be  absent  or  to  not  have  been  considered  such  as  the  need  for 
capability  management  of  SoS  engineering  (SoSE).  This  absence  causes  confusion 
between  capabilities  and  requirements  and  between  evolutionary  acquisitions  and  its 
development  processes,  and  results  in  no  significant  operational  change  in  space  system 
procurement.  Technology  issues  were  also  discussed — specifically  the  debate  between 
using  evolutionary  or  revolutionary  strategies  to  incorporate  technology.  The  literature 
review  closed  with  a  brief  explanation  of  how  DoD  and  the  commercial  world  define  an 
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evolutionary  strategy  and  system  development  strategies  to  develop  the  technical 
composition  of  a  system. 
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III.  Methodology 


3.1  Introduction 

This  chapter  will  address  the  qualitative  procedure,  the  research  design  type,  the 
researcher’s  role,  the  data  collection  procedures,  the  data  analysis  procedures,  and  the 
steps  taken  to  verify  the  research. 

3.2  A  Qualitative  Procedure 

The  format  of  this  chapter  has  been  closely  modeled  after  CreswelTs  account  of  a 
qualitative  procedure  in  Chapter  Nine  of  his  book  titled  Research  Design:  Qualitative 
and  Quantitative  Approaches  (Creswell,  1994).  This  methodology  will  describe  a 
qualitative  procedure,  the  case  study  design,  the  researcher’s  role,  the  data  collection 
procedures,  the  data  analysis  procedures,  and  the  steps  taken  to  verify  the  research. 

3.2.1  Why  Qualitative? 

The  purpose  of  this  research  is  to  identify  how  effective  the  integrated 
deployment  of  the  CJCSI  3170  and  the  NSSAP  03-01  has  been  for  national  space 
systems.  This  problem  requires  a  qualitative  approach  for  a  number  of  reasons, 
supported  by  Morse. 

Characteristics  of  a  qualitative  research  problem  are:  (a)  the 
concept  is  ‘immature’  due  to  a  conspicuous  lack  of  theory 
and  previous  research;  (b)  a  notion  that  the  available  theory 
may  be  inaccurate,  inappropriate,  incorrect,  or  biased;  (c)  a 
need  exists  to  explore  and  describe  the  phenomena  and  to 
develop  theory;  or  (d)  the  nature  of  the  phenomenon  may 
not  be  suited  to  quantitative  measures  (Morse,  1991). 

Such  a  problem  as  the  one  examined  in  this  research  fits  a  number  of  the  above 
characteristics  in  one  way  or  another,  but  most  significantly  fits  characteristics  a  and  c. 
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As  the  literature  review  in  chapter  two  has  shown,  there  is  very  little  scientific  research 
that  has  been  conducted  on  the  matter.  While  some  literature  on  product  development  is 
available,  usually  pertaining  to  the  commercial  world,  the  overall  concept  of  identifying 
military  deficiencies  and  translating  those  into  solutions  is  extremely  limited.  Much  of 
the  literature  looking  at  this  problem  seems  to  focus  on  identifying  broad  problems  with 
the  entire  system  and  lacks  a  validated  proposal  of  theory  or  solutions,  and  is  limited  to 
the  authors’  experiences. 

As  for  characteristic  b,  this  research  makes  no  suggestion  that  the  existing  process 
and  acquisition  theory  is  “inaccurate,  inappropriate,  incorrect,  or  biased.”  Instead,  it 
attempts  to  simply  identify  the  effectiveness  of  implementation  at  this  point  for  three 
space  programs  and  if  applicable,  identify  areas  for  improvement. 

Due  to  the  complexity  of  the  process  and  the  need  for  more  research  on  the 
subject,  this  research  is  not  suited  for  a  quantitative  approach  at  this  time — not  to  say  that 
a  quantitative  approach  will  never  be  applicable.  Quantitative  approaches  are  excellent 
for  removing  or  accounting  for  biases  and  assumptions,  if  those  are  known  and  can  be 
controlled. 

3.2.2  The  Assumptions  of  Qualitative  Designs 

Although  a  qualitative  approach  is  definitely  appropriate  for  the  proposed 
problem,  it  is  important  to  understand  the  assumptions  that  accompany  such  an  approach. 
Merriam  describes  those  as  follows. 

1.  Qualitative  researchers  are  concerned  primarily  with 
process,  rather  than  outcomes  or  products. 

2.  Qualitative  researchers  are  interested  in  meaning — how 
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people  make  sense  of  their  lives,  experiences,  and  their 
structures  of  the  world. 

3.  The  qualitative  researcher  is  the  primary  instrument  for 
data  collection  and  analysis.  Data  are  mediated  through 
this  human  instrument,  rather  than  through  inventories, 
questionnaires,  or  machines. 

4.  Qualitative  research  involves  fieldwork.  The  researcher 
physically  goes  to  the  people,  setting,  site,  or  institution  to 
observe  or  record  behavior  in  its  natural  setting. 

5.  Qualitative  research  is  descriptive  in  that  the  researcher 
is  interested  in  process,  meaning,  and  understanding  gained 
through  words  or  pictures. 

6.  The  process  of  qualitative  research  is  inductive  in  that 
the  researcher  builds  abstractions,  concepts,  hypotheses, 
and  theories  from  details  (Merriam,  1988). 

These  assumptions  will  be  readily  apparent  in  the  next  section  describing  the 

design  type. 

3.3  The  Type  of  Design 


3.3.1  Specific  Qualitative  Design 

There  are  a  number  of  research  strategies  that  apply  to  problems  involving  the 
social  sciences.  Determining  which  strategy  to  use  depends  on  the  form  of  the  research 
question,  the  researcher’s  ability  to  control  surrounding  events,  and  whether  or  not  the 
focus  is  on  contemporary  events  (Yin,  2003).  For  the  problem  proposed,  a  case  study 
will  best  answer  the  question  because  a  case  study  answers  how  and  why  questions,  does 
not  require  control  of  the  environment,  and  focuses  on  contemporary  events  (Yin,  2003). 
In  contrast,  an  experiment  could  answer  the  same  questions  and  focus  on  contemporary 
events  as  well,  but  would  require  control  of  the  environment.  A  historical  strategy  could 
also  answer  the  same  questions  and  not  require  control  of  the  environment,  but  would  not 
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focus  on  contemporary  events.  These  methods  can  be  quantitative  or  qualitative; 
however  for  the  reasons  mentioned  in  the  previous  section,  the  case  study  will  be 
qualitative  in  its  application  to  this  problem.  Yin  (2003)  defines  case  studies  with  two 
propositions: 


1 .  A  case  study  is  an  empirical  inquiry  that 

•  investigates  a  contemporary  phenomenon  within  its  real-life  context,  especially 
when 

•  the  boundaries  between  phenomenon  and  context  are  not  clearly  evident 

2.  The  case  study  inquiry 

•  copes  with  the  technically  distinctive  situation  in  which  there  will  be  many 
more  variables  of  interest  than  data  points,  and  as  one  result 

•  relies  on  multiple  sources  of  evidence,  with  data  needing  to  converge  in  a 
triangulating  fashion,  and  as  another  result 

•  benefits  from  the  prior  development  of  theoretical  propositions  to  guide  data 
collection  and  analysis  (Yin,  2003) 


3.3.2  Characteristics  of  the  Design 

Case  studies  have  five  important  components  (Yin,  2003)  that  will  be  discussed  in 
the  context  of  the  problem  this  research  addresses.  Those  components  are  a  study’s 
questions,  propositions,  unit  of  analysis,  links  between  the  data  and  propositions,  and  the 
criteria  for  interpreting  the  findings. 

Study  Questions.  The  study  question  is  the  focus  of  the  research.  In  this  case,  the 
research  attempts  to  answer  how  effective  the  integrated  deployment  of  the  CJCSI  3170 
and  the  NS  SAP  03-01  has  been  for  national  space  systems. 

Study  Propositions.  Propositions  direct  the  focus  of  research  and  are  frequently 
present  in  social  science  research,  unless  the  purpose  of  the  research  is  exploratory.  In 
that  case,  such  propositions  are  unnecessary  because  the  exploration  topic  has  already 
been  selected  and  does  not  require  focus.  There  are  three  propositions  within  this 
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research  effort.  Those  propositions  are:  (1)  organizations  at  different  levels  in  the  process 
may  have  a  different  understanding  of  what  the  new  guidance  proposes;  (2)  space 
systems,  following  the  same  guidance,  may  implement  the  guidance  differently;  (3)  the 
decision  process  required  for  such  guidance  to  be  effective  is  inadequate. 

Unit  of  Analysis.  The  unit  of  analysis  to  be  studied  is  each  stake  holding 
organization  in  the  approval  chain  for  a  new  system  to  be  developed  or  an  existing 
system  to  be  modified,  based  on  an  identified  capability  deficiency.  These  organizations 
will  be  identified  in  the  context  of  three  space  systems,  making  this  a  multiple  case  study. 

The  last  two  components,  Linking  Data  to  Propositions  and  Criteria  for 
Interpreting  the  Findings  will  be  discussed  as  Data  Analysis  Procedures  in  section  3.6. 

3.4  The  Researcher’s  Role 

3.4.1  Past  Experiences  of  Researcher 

In  June  of  2002,  the  researcher  was  assigned  to  the  MILSATCOM  (Military 
Satellite  Communications)  Joint  Program  Office  (MJPO).  The  Joint  Program  Office  had 
a  number  of  interesting  roles  in  addition  to  being  a  traditional  System  Program  Office 
(SPO).  The  organization  was  comprised  of  personnel  from  the  Air  Force,  Anny,  and 
Navy,  reflecting  their  joint  interest  in  the  mission  and  success  of  the  programs.  MJPO 
did  not  house  a  single  satellite;  rather,  the  program  office  was  responsible  for  the 
management  of  a  number  of  satellite  programs  in  various  stages  of  development,  co¬ 
located  at  Los  Angeles  AFB  and  Hanscom  AFB.  Additionally,  the  MJPO  managed  the 
acquisition  of  the  necessary  terminals  to  communicate  with  the  satellites,  being 
developed  at  Hanscom  AFB.  A  program  office  managing  both  the  satellites  and  their 
respective  tenninals  was  traditionally  referred  to  as  a  “basket  SPO.”  Today,  that  scenario 
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is  more  frequently  described  as  an  office  managing  a  system-of-systems  (SoS). 
Similarly,  a  program  office  managing  a  number  of  similar  functioning  systems,  such  as 
communication  satellites  (although  they  ultimately  perfonn  different  missions  at  different 
frequencies),  is  said  to  be  an  office  managing  a  family-of-systems  (FoS). 

3.4.2  Selection  Steps 

Understanding  the  acquisition  process,  the  author  identified  a  number  of 
organizations  to  interview  both  vertically  and  horizontally.  By  vertical,  organizations 
were  selected  along  an  approving  chain  of  command  from  top  to  bottom,  requirements 
generation  to  operator  use.  In  this  manner,  data  could  be  analyzed  at  all  phases  of  an 
acquisition  program.  By  horizontal,  the  author  asked  the  same  questions  at  all  levels,  for 
three  different  space  programs.  This  would  indicate  if  policies  were  implemented  to  the 
same  extent,  or  documented  for  exceptions,  throughout  space  programs.  Programs  in 
different  phases  of  the  acquisition  process,  meaning  that  some  were  initiated  before  and 
after  the  policy  change,  were  selected  to  maximize  testing  the  extent  of  implementation. 

Both  organizations  and  programs  selected  for  research  were  selected  based  on 
criteria  that  would  lead  to  diversified  results  enabling  comparison.  Specific  interview 
participants  were  selected  based  on  the  author’s  contacts  and  “snowball  sampling” — 
asking  the  interview  participants  who  they  thought  should  be  contacted.  Although 
snowball  sampling  is  considered  to  have  a  number  of  biases  when  studying  social 
interactions,  those  biases  are  mitigated  by  the  fact  that  social  relationships  were  not  being 
studied  and  outweighed  by  the  value  that  snowball  sampling  can  provide  when  it  is 
difficult  to  find  interview  participants. 
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3.4.3  Human  Subjects  and  Ethical  Issues 


Interview  participants  can  be  difficult  to  find  for  a  number  of  reasons,  but  in  this 
particular  case,  one  reason  may  be  the  fear  of  retribution  resulting  in  negative 
consequences  since  interview  participants  are  being  asked  to  comment  on  the 
effectiveness  of  policies  created  by  superiors.  For  this  reason,  all  personal  information 
will  remain  completely  confidential  throughout  the  entire  process,  including  thesis 
publication  and  any  related  articles  that  may  result.  Additionally,  the  research  was 
approved  through  the  Air  Force  Institute  of  Technology’s  Human  Subject  Research 
Approval  process.  This  infonnation  was  provided  to  the  interview  participant  in  an 
interview  invitation  letter  and  before  the  interview  was  actually  conducted.  The 
interview  invitation  letter  is  located  in  Appendix  A. 

3.5  Data  Collection  Procedures 

3.5.1  Parameters  of  Data  Collection 

Individuals  from  all  levels  of  the  Joint  Capability  Integrated  Development  System 
(JCIDS)  and  the  National  Security  Space  Acquisition  Process  (NSSAP),  as  described  in 
Figure  2.02,  were  interviewed  on  location,  whenever  possible.  Specifically,  individuals 
have  been  identified  from  the  following  offices. 

•  National  Security  Space  Office  (NSSO) 

•  Secretary  of  the  Air  Force  Directorate  of  Space  Acquisitions  (SAF/USA) 

•  Headquarters  Air  Force  Deputy  Chief  of  Staff  for  Air  &  Space  Operations 
(AF/XO) 

•  Headquarters  Air  Force  Space  Command  Directorate  of  Requirements 
(AFSPC/A5) 

•  Global  Positioning  System  (GPS)  Program  Office,  System  Engineers 

•  Space  Based  Infrared  System  (SBIRS)  Program  Office,  System  Engineers,  Users 

•  Transformational  Communications  Satellite  (TSAT)  Program  Office,  System 
Engineers 
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•  Aerospace  Corporation 

•  Space  and  Missile  Systems  Center  (SMC)  Acquisition  Center  of  Excellence 

(ACE) 

•  Defense  Systems  Management  Course  (DSMC)  Instructors 

•  Air  Force  Institute  of  Technology  (AFIT)  Instructors 

•  Defense  Acquisition  University  (DAU)  Instructors 

The  questions  were  focused  on  these  individuals’  roles  in  the  acquisition  process, 
before  and  after  the  implementation  of  CJCSI  3170  and  NSSAP  03-01,  as  well  as  how 
they  view  the  roles  of  others.  The  questions  also  focused  on  the  implementation  of  the 
policies. 

3.5.2  Types  of  Data  Collected,  Rationale,  and  Protocol 

The  interviews  were  comprised  of,  but  not  limited  to,  a  standard  set  of  questions 
located  below  and  again  in  Appendix  B.  The  semi-structured  format  of  the  interview 
provided  both  structure  for  analytical  purposes,  as  well  as  the  flexibility  to  ask  follow  up 
questions  for  clarity  that  may  be  applied  to  understanding  a  specific  organization’s  role  or 
perspective. 

The  standard  script  included  a  heading,  instructions,  standard  research  questions, 
and  participant-specific  and/or  follow-up  questions.  The  interview  participant’s 
responses  were  recorded,  whenever  possible  via  digital  tape  recorder.  However,  if  the 
interview  participant  preferred,  they  were  able  to  complete  the  interview  through  an  e- 
mail  dialogue.  Although,  differences  in  data  collection  method  are  generally  not 
preferred  (unless  analyzing  the  method),  the  value  in  the  collection  of  responses  from 
various  organizations  outweighs  the  bias  that  may  be  introduced  through  the  difference  in 
collection  method.  All  recorded  interviews  were  transcribed  into  an  interview  record. 
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Additionally,  the  interviewer,  in  this  case  the  author,  took  written  notes  to  indicate  a 
record  of  points  of  interest  or  points  requiring  follow-up. 

3.6  Data  Analysis  Procedures 

Like  many  other  aspects  of  qualitative  case  study  research,  there  is  no  “right  way” 
to  analyze  data  (Tesch,  1990;  Creswell,  1994);  rather,  there  is  no  one  right  way.  There 
are  a  number  of  accepted  guidelines  that  can  ease  the  emergence  of  themes,  patterns,  and 
theories.  Creswell  (1994)  suggests  a  spiral  model,  aptly  appropriate  considering  the 
subject  matter,  in  which  raw  data  is  continually  reviewed  with  a  different  focus  each  time 
refining  the  data  and  making  it  usable.  These  categories  and  a  brief  description  of  each 
are  identified  in  Figure  3.01.  Tesch  (1990)  describes  a  similar,  more  detailed  process  in 
eight  steps  listed  below. 


1.  Get  a  sense  of  the  whole.  Read  through  all  of  the 
transcripts  carefully.  Perhaps  jot  down  some  ideas  as  they 
come  to  mind. 

2.  Pick  one  document  (one  interview) — the  most 
interesting,  the  shortest,  the  one  on  the  top  of  the  pile.  Go 
through  it  asking  yourself,  What  is  this  about?  Do  not  think 
about  the  “substance”  of  the  information,  but  rather  its 
underlying  meaning.  Write  thoughts  in  the  margin. 

3.  When  you  have  completed  this  task  for  several 
informants,  make  a  list  of  all  topics.  Cluster  together 
similar  topics.  Fonn  these  topics  into  columns  that  might 
be  arrayed  as  major  topics,  unique  topics,  and  leftovers. 

4.  Now  take  this  list  and  go  back  to  your  data.  Abbreviate 
the  topics  as  codes  and  write  the  codes  next  to  the 
appropriate  segments  of  the  text.  Try  out  this  preliminary 
organizing  scheme  to  see  whether  new  categories  and  codes 
emerge. 
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5.  Find  the  most  descriptive  wording  for  your  topics  and 
turn  them  into  categories.  Look  for  reducing  your  total  list 
of  categories  by  grouping  topics  that  relate  to  each  other. 
Perhaps  draw  lines  between  your  categories  to  show 
interrelationship  s . 

6.  Make  a  final  decision  on  the  abbreviation  for  each 
category  and  alphabetize  the  codes. 

7.  Assemble  the  data  material  belonging  to  each  category  in 
one  place  and  perfonn  a  preliminary  analysis. 

8.  If  necessary,  recode  your  existing  data.  (Tesch,  1990) 


Figure  3.01  -  Creswell’s  Data  Analysis  Spiral  (after  Creswell,  1994) 
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These  processes  reflect  Dr.  Kisenhardt’s  (1989)  position  that  case  studies  can  be 
used  to  build  theories  if  they  remain  open-ended.  Consequently,  developing  a  more 
structured  analysis  is  difficult  without  knowing  the  results  of  such  data  collection.  In  this 
case,  data  analysis  occurs  simultaneously  with  data  collection,  data  interpretation,  and 
report  writing  (Creswell,  1994).  Therefore,  data  analysis  consists  of  recognizing  themes 
that  may  indicate  how  effective  the  integrated  deployment  of  the  CJCSI  3170  and  the 
NSSAP  03-01  has  been  for  national  space  systems. 

3.7  Verification  Steps 

Verification  is  often  considered  an  indication  of  truly  scientific  research 
(Creswell,  1994)  but  is  difficult  to  formalize  in  qualitative  research.  Consider  some  of 
the  assumptions  and  desired  results  of  a  qualitative  process.  The  research  is  usually 
descriptive  of  a  process  that  the  researcher  has  limited  access  to;  in  other  words,  the 
researcher  can  not  examine  every  single  case  at  every  point  in  time  with  everyone 
involved.  The  research  is  also  inductive  in  that  the  researcher  frequently  uses  it  to  build 
hypotheses,  theories,  and  models  to  be  tested. 

With  this  in  mind,  it  is  understandable  that  some  researchers  suggest  using  terms 
such  as  “authenticity”  and  “trustworthiness”;  however  both  Creswell  (1994)  and  Miles 
and  Hubennan  (1994)  suggest  using  terms  of  validity  and  reliability.  Miles  and 
Huberman  have  also  compiled  a  number  of  reflective  questions  that  are  useful  in 
assessing  validity  and  reliability.  Some  of  Miles  and  Huberman’ s  questions  are  listed 
below  that  will  be  used  in  this  research. 
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3. 7.1  Internal  Validity 


Creswell  and  Merriam  define  internal  validity  as  “the  accuracy  of  the  infonnation 
and  whether  it  matches  reality.”  Internal  validity  will  be  achieved  by  providing  a  number 
of  the  interview  participants  with  the  research  results  and  asking  them  to  comment  on 
their  accuracy  in  addition  to  a  review  of  Miles  and  Huberman’s  questions. 

3. 7.2  External  Validity 

External  validity  or  transferability  or  fittingness  indicates  how  “generalizable”  the 
conclusions  of  a  study  may  be  (Miles  and  Huberman,  1994).  There  appears  to  be  some 
difference  in  opinion  as  far  as  whether  it  is  good  to  have  a  large  or  a  small  amount  of 
generalizability.  According  to  Merriam,  “the  intent  of  qualitative  research  is  not  to 
generalize  findings,  but  to  form  a  unique  interpretation  of  event.”  Miles  and  Hubennan 
take  a  more  balanced  approach,  commenting  that  some  generalizability  is  good  because 
there  is  meaning  from  the  research  for  more  people.  However,  if  the  findings  are  too 
general,  they  begin  to  lose  their  importance. 

3. 7.3  Reliability 

According  to  Miles  and  Huberman  (1994),  “the  underlying  issue  here  is  whether 
the  process  of  the  study  is  consistent,  reasonably  stable  over  time  and  across  researchers 
and  methods.”  This  research  builds-in  reliability  by  examining  three  different  space 
programs,  using  the  same  protocol.  “In  case  study  research,  in  which  the  investigator 
explores  multi-site  cases,  one  can  examine  whether  the  same  patterns  or  events  or 
thematic  constructs  are  replicated  in  different  settings”  (Creswell,  1994). 


53 


Table  3.01  -  Miles  and  Huberman  ’s  Validity  and  Reliability  Questions  (Miles  & 
Huberman,  1994) 


Internal  Validity  / 
Credibility  /  Authenticity 

External  Validity  / 
Transferability  / 

Fittingness 

Reliability  /  Dependability 
/  Auditability 

2.  Does  the  account  “ring 
true,”  make  sense,  seem 
convincing  or  plausible, 
enable  a  “vicarious 
presence”  for  the  reader? 

1 .  Are  the  characteristics  of 
the  original  sample  of 
persons,  settings,  processes 
(etc.)  fully  described 
enough  to  permit  adequate 
comparisons  with  other 
samples? 

1 .  Are  the  research 
questions  clear,  and  are  the 
features  of  the  study  design 
congruent  with  them? 

5.  Are  the  presented  data 
well  linked  to  the  categories 
of  prior  or  emerging  theory? 
Do  the  measures  reflect  the 
constructs  in  play? 

3.  Is  the  sampling 
theoretically  diverse  enough 
to  encourage  broader 
applicability? 

2.  Is  the  researcher’s  role 
and  status  within  the  site 
explicitly  described? 

8.  Are  areas  of  uncertainty 
identified?  (There  should 
be  some.) 

5.  Do  the  findings  include 
enough  “thick  description” 
for  readers  to  assess  the 
potential  transferability, 
appropriateness  for  their 
own  settings? 

3.  Do  findings  show 
meaningful  parallelism 
across  data  sources 
(infonnants,  contexts, 
times)? 

10.  Have  rival  explanations 
been  actively  considered? 
What  happened  to  them? 

6.  Does  a  range  of  readers 
report  the  findings  to  be 
consistent  with  their  own 
experiences? 

4.  Are  basic  paradigms  and 
analytic  constructs  clearly 
specified? 

12.  Were  the  conclusions 
considered  to  be  accurate  by 
original  informants?  If  not, 
is  there  a  coherent 
explanation  for  this? 

7.  Are  the  findings 
congruent  with,  connected 
to,  or  confirmatory  of  prior 
theory? 

5.  Were  data  collected 
across  the  full  range  of 
appropriate  settings,  times, 
respondents,  and  so  on 
suggested  by  the  research 
questions? 

1 1 .  Does  the  report  suggest 
settings  where  the  findings 
could  fruitfully  be  tested 
further? 

8.  Were  data  quality  checks 
made  (e.g.  for  bias,  deceit, 
informant 
knowledgeability)? 

12.  Have  the  findings  been 
replicated  in  other  studies  to 
assess  their  robustness?  If 
not,  could  replication  efforts 
be  mounted  easily? 

9.  Do  multiple  observers’ 
accounts  converge,  in 
instances,  settings,  or  times 
when  they  might  be 
expected  to? 

10.  Were  any  forms  of  peer 
or  colleague  review  in 
place? 
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3.8  Summary 


In  this  chapter,  the  author  addressed  various  aspects  of  a  qualitative  procedure, 
specifically  why  it  was  selected  for  this  research  and  the  assumptions  that  must 
accompany  a  qualitative  procedure.  The  multiple  case  study  was  announced  as  the 
research  design  type.  The  definition  of  a  case  study  and  described  characteristics  of  such 
a  design  type  indicated  the  appropriateness  of  selection  for  answering  the  research 
question:  how  effective  the  integrated  deployment  of  the  CJCSI  3170  and  the  NSSAP  03- 
01  has  been  for  national  space  systems.  The  researcher’s  role  was  described  within  the 
context  of  the  researcher’s  related  experience  before  the  research,  and  the  researcher’s 
role  of  selecting  cases  and  participants.  The  data  collection  procedure  and  analysis 
procedures  were  also  described.  Data  collection  consisted  of  horizontal  and  vertical 
interviews  from  a  number  of  organizations  across  multiple  programs  and  with  differing 
approval  authority.  Data  analysis  was  completed  by  drawing  themes  from  multiple 
reviews  of  the  data.  The  research  was  verified  with  measures  to  address  internal  and 
external  reliability,  in  addition  to  reliability.  The  results  of  this  methodology  are 
discussed  in  Chapters  Four  and  Five. 
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IV.  Results  and  Discussion 


4.1  Introduction 

In  addition  to  reviewing  relevant  research,  articles,  commission  reports,  and 
guiding  documentation,  interviews  were  conducted  across  a  number  of  organizations 
involved  in  the  acquisition  process.  These  organizations  included  system  acquirers, 
users,  air  and  space  requirement  developers,  architecture  designers,  space  policy 
directors,  and  experts  from  the  Defense  System  Management  Course,  Defense 
Acquisition  University,  and  Air  Force  Institute  of  Technology.  Due  to  the  policies  of  the 
Investigative  Research  Board,  the  specific  comments  can  not  be  published  if  they  can  be 
attributed  to  an  individual.  Since  individuals,  frequently  used  personal  examples  and 
were  guaranteed  confidentiality,  the  results  of  analysis  are  captured  through  the  answers 
to  the  investigative  questions. 

Recommendations  are  included  in  the  investigative  questions  and  in  the  following 
section.  Some  investigative  questions  were  answered  through  another  question’s 
response  and  do  not  have  a  separate  answer. 

4.2  Investigative  Question 

What  are  the  goals  of  an  EA  strategy?  The  goal  of  an  evolutionary  acquisition 
strategy  is  to  reduce  the  amount  of  time  it  takes  to  get  added  appropriate  capability  to  the 
user  and  to  reduce  cost  growth.  This  is  done  by  employing  the  strategy  described  as 
follows. 
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What  is  an  evolutionary  acquisition  (EA)  strategy?  To  the  Department  of  Defense 
space  community,  an  evolutionary  acquisition  strategy  describes  the  acquisition  of  a 
number  of  blocks  of  similar  items  with  increasing  capability.  The  added  capability  is 
planned  two  to  three  blocks  earlier  than  the  block  that  the  capability  will  be  incorporated 
in.  Reasons  for  added  capability  may  be  a  changing  threat  or  a  change  in  available 
technology.  This  is  especially  important  since  DoD  does  not  drive  technological 
development  like  it  did  50  years  ago  (Brown,  2004).  Although,  this  is  more  visible 
outside  of  space  systems,  the  development  of  Spaceship  One  and  plans  in  Virginia  and 
Florida  for  space  tourism  are  examples  of  an  outside  and  commercial  driver. 

While  this  appears  to  produce  similar  systems  to  those  produced  in  the  past  (i.e. 
blocks  with  increasing  capability),  there  is  a  different  emphasis  in  how  technology  is 
planned.  A  100%  solution  is  no  longer  sought  for  in  the  first  block,  with  additional 
capability  added  in  following  blocks.  Instead,  the  first  block  should  be  about  an  80% 
solution  and  must  provide  an  improved  capability  over  the  current  system  according  to 
the  definition  in  NSSAP  03-01,  or  else  not  approved  until  mature  technology  is  available. 
However,  some  interviewees  stated  that  among  the  user  community,  there  is  a  fear  that  an 
initial  block  will  be  developed  to  lay  the  groundwork  for  the  following  block  to  “leap 
frog”  over  the  capability  of  the  former  system,  but  the  initial  block  will  actually  take  a 
step  back  in  capability.  Or,  that  the  initial  block  may  be  such  a  slight  improvement  that 
the  second  block  will  not  be  procured  because  the  value  to  cost  ratio  was  so  low.  To  the 
first  problem,  this  is  by  NSSAP  03-01  definition,  not  an  evolutionary  acquisition.  To  the 
second  problem,  if  the  increment  was  achieved  with  the  cost  that  was  budgeted,  then 
there  would  be  no  change  in  perceived  value  between  the  time  the  system  was  approved 
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and  the  first  block  was  completed.  This  is  possible  if  fairly  mature  technology  is 
incorporated  from  the  beginning,  because  cost  estimates  will  be  more  accurate.  In 
addition  to  incorporating  mature  technology  to  reduce  cost,  EA  attempts  to  reduce  cost  by 
reducing  risk  through  the  selection  of  the  development  process.  This  is  discussed  in  a 
later  section. 

Because  an  80%  solution  means  that  there  is  at  least  20%  more  coming,  and  even 
more  through  technology  improvements  where  applicable,  there  is  an  increased  need  to 
make  arrangements  up  front  to  allow  for  this  capability  to  be  added.  The  space 
community  understands  this  well  because  space-based  systems  can  not  cost  effectively  be 
upgraded  once  launched,  at  this  time.  Therefore,  any  incremental  improvements  to  a 
space  system  have  traditionally  been  achieved  through  ground-based  software  and 
hardware  upgrades  for  tenninals  or  by  launching  backward  compatible  satellites  to 
supplement  an  existing  constellation. 

None  of  the  interviewees  viewed  an  evolutionary  acquisition  strategy  as  a  new 
strategy  for  the  development  of  space  systems.  In  truth,  as  defined  in  NSSAP  03-01, 
many  of  today’s  space  systems  could  have  operated  under  an  evolutionary  strategy.  But, 
the  results  may  have  been  different  if  full  understanding  of  the  goals  of  EA  was  achieved. 
For  example,  interviewees  were  asked  to  comment  on  the  current,  and  past  where 
applicable,  acquisition  strategies  for  GPS,  SBIRS,  and  TSAT.  Several  interviewees 
referred  to  the  initial  acquisition  strategy  for  all  three  programs  as  the  “Big  Bang” — 
meaning  that  the  programs  tried  to  give  the  user  all  the  capability  that  was  requested  in 
the  initial  block.  Some  program  offices  seemed  to  realize  sooner  than  others  that  this 
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could  not  be  achieved  within  cost  and  schedule  constraints  and  incorporated  a  more 
evolutionary,  rather  than  revolutionary,  strategy. 

GPS  was  initiated  before  an  EA  strategy  was  the  preferred  strategy  and  used 
block  development  to  procure  the  first  blocks  and  models  of  the  satellite.  At  the  time,  the 
strategy  was  not  considered  evolutionary,  but  additional  capability  was  planned  for 
following  blocks  from  program  initiation.  GPS  is  now  considered  to  be  an  evolutionary 
acquisition  and  the  third  block  will  employ  spiral  development. 

SBIRS  was  also  initiated  before  an  EA  strategy  was  the  preferred  strategy.  The 
traditional  waterfall  model  was  used  to  design  the  system  and  the  system  was  acquired 
through  a  process  known  as  Total  System  Performance  Responsibility  (TSPR).  TSPR 
was  created  to  reduce  lengthy  and  cumbersome  government  oversight  and  allow  the 
contractor  more  flexibility  to  use  best  commercial  practices  to  develop  a  system  as  long 
as  the  overall  objectives  were  met.  When  the  SBIRS  program  first  breached  its 
Acquisition  Program  Baseline  (APB),  a  decision  was  made  to  switch  oversight  from  the 
contractor  back  to  the  government.  Additional  requirements  were  levied  on  the  SBIRS 
program  office  and  the  acquisition  of  SBIRS  began  to  take  a  more  evolutionary  form  in 
development,  but  not  in  acquisition.  Necessary  acquisition  and  operational  level  tasks 
were  broken  into  “effectivities;”  however,  the  completion  of  one  effectivity  did  not  add 
an  improved  capability  because  the  effectivity  was  only  effective  if  the  whole  system  was 
operational.  There  are  a  number  of  differing  opinions  within  SBIRS  as  to  whether  an 
evolutionary  acquisition  strategy,  as  defined  in  NSSAP  03-01,  was  ever  used. 

The  concept  for  TSAT  was  initiated  slightly  before  EA  became  the  preferred 
strategy  and  one  interviewee  said  that  it  would  have  been  an  incremental  development 
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process  anyway.  TSAT’s  use  of  an  evolutionary  acquisition  strategy  employing 
incremental  development  was  done  “to  show  reduction  of  risk  through  early  launch.” 

The  space  community’s  employment  of  an  EA  strategy  has  not  achieved  the 
desired  goals  for  at  least  two  reasons.  The  80%  solution  has  not  really  been  adopted  as 
an  acceptable  solution,  probably  because  of  some  of  the  concerns  described  above.  The 
second  reason  is  that  the  space  community  is  still  looking  for  a  revolutionary 
advancement.  Transformation  seems  to  be  viewed  more  as  a  radical  change  than  small 
evolving  changes  to  modernize.  Even  if  employed  correctly,  an  EA  strategy  has  many 
new  ramifications  given  the  transformed  environment  that  space  systems  are  operating  in. 
This  will  be  discussed  in  following  sections  on  capability-acquisitions. 

How  is  an  EA  strategy  implemented?  The  technological  portion  of  an  acquisition 
strategy  is  implemented  through  a  development  process.  For  DoD  space  systems,  the 
practical  difference  between  spiral  or  incremental  development  is  somewhat  gray,  but  all 
of  the  interviewees  differentiated  the  terms  correctly  by  the  degree  to  which  the  desired 
end  state  of  a  system  is  known.  However,  the  development  process  occurs  within  a 
block,  according  to  Figure  2.07,  and  the  desired  end  state  of  the  block  that  is  being 
developed  should  be  known.  Therefore  the  distinguishing  characteristic  of  degree  to 
which  the  end  state  is  known,  does  not  really  make  sense  when  considering  a  block. 
Similarly,  if  considering  the  system  instead  of  the  block,  is  the  desired  end  state  ever 
known  if  the  purpose  of  EA  is  to  allow  for  changing  threats  or  technology?  This 
differentiation  between  development  processes  should  be  removed  and  one  process,  such 
as  the  spiral-mapped  Vee  model  could  be  used  to  develop  a  block.  The  spiral-mapped 
Vee  model  is  the  same  model  depicted  on  the  Integrated  Defense  Acquisition, 
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Technology,  and  Logistics  Life  Cycle  Management  Framework  from  2004.  This  map  is 
located  in  the  back  cover  of  the  thesis. 

There  is  more  to  an  acquisition  strategy  than  the  technical  development  aspect. 
There  are  contractual,  financial,  testing,  and  logistical  implications  as  well  that  really 
have  not  been  considered  in  NSSAP  03-01.  When  asked  to  comment  on  evolutionary 
acquisition,  one  interviewee  colorfully  responded  “What’s  a  spiral  look  like?  A  screw — 
and  that’s  what  it  means  for  logistics.” 

As  shown  in  the  above  statement,  it  is  not  uncommon  for  the  terms  evolutionary 
acquisitions  and  spiral  development  to  be  interchanged,  however  all  of  the  academic- 
based  (meaning  AFIT,  DAU,  and  DSMC)  interviewees  could  distinguish  between  the 
tenns,  while  users  and  some  program  office  interviewees  had  more  difficulty.  This  may 
be  attributed  to  the  fact  that  the  terms  evolutionary,  spiral,  and  incremental  are  used 
almost  interchangeably  (Larman,  2003)  in  commercial  organizations  and  in  particular, 
software  design  and  systems  engineering  communities.  These  specifically  describe 
development  processes,  with  the  exception  of  software  engineering  which  can  produce 
useful  code  after  one  spiral.  Perhaps  a  worthy  metaphor  for  clarification  of  what  DoD’s 
EA  strategy  means  even  for  software  intensive  space  systems  is  the  term  “evolutionary 
deployment.”  Figure  4.01  depicts  evolutionary  deployment  and  development. 
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Figure  4.01  -  Evolutioncuy  Deployment  v.  Development  (Gamache,  2006) 


Evolutionary  Deployment 

(Evolutionary  Acquisition) 


Concept 

Decision 


What  development  processes  are  alternatives  to  evolutionary  acquisitions? 
Development  processes  are  not  alternatives  to  acquisition  strategies.  As  mentioned 
above,  they  represent  the  technical  portion  of  a  system.  An  alternative  to  an  EA  strategy 
would  be  one  in  which  all  desired  capability  was  achieved  in  one  block.  This  alternative 
could  even  use  spiral  development  and  not  be  an  evolutionary  acquisition  system. 
Alternative  development  processes  were  described  in  Chapter  Two.  However,  several 
systems  engineering  experts  recommended  that  the  Vee  be  used  because  it  is  similar  to 
the  traditional  waterfall  model  if  used  once  or  similar  to  a  spiral  if  repeated  as  more 
information  about  the  desired  capability  is  available. 
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What  is  a  capability-based  acquisition?  There  was  a  major  difference  in  how  the 
tenn  capability-based  acquisition  was  defined  between  literature  and  interviewees. 
Literature  tended  to  define  the  tenn  as  requirements  and  acquisition  processes  focused  on 
meeting  a  desired  capability  through  a  number  of  means,  including  Family-of-Systems, 
System-of-Systems,  or  a  traditional  platform.  This  concept  was  supported  by  the 
emphasis  on  interoperable  systems  for  joint  users,  especially  for  information  systems 
including  space  systems.  Interviewees  defined  capability-based  acquisitions  as  a 
requirements  process  that  was  more  relaxed  in  the  extent  to  which  requirements  were 
identified  or  defined,  with  the  goal  of  giving  the  contractor  more  flexibility  in  how  a 
system  is  developed.  These  two  ideas  are  almost  mutually  exclusive.  The  literature 
definition  is  supported  in  policy,  while  the  interviewees’  definition  is  in  sharp  contrast  to 
findings  of  several  commissions  investigating  acquisition  problems  over  the  past  twenty 
years.  Specifically,  the  interoperability  that  is  necessary  to  achieve  a  joint  netcentric 
environment  requires  adherence  to  specific  standards  in  several  areas. 

The  literature  definition  of  capability-based  acquisitions  has  a  significant  impact 
on  an  evolutionary  acquisition  strategy.  Not  only  do  program  managers  have  to  manage 
a  number  of  evolving  blocks,  making  trade-offs  between  blocks  to  reduce  risk  or 
incorporate  new  technology  or  address  a  different  threat.  Now  they  must  make  these 
decisions  considering  the  affects  to  other  systems  within  a  program’s  FoS  or  SoS. 
Disciplined  systems  engineering  practices  are  more  important  than  ever  to  make  such 
vertical  (between  blocks)  and  horizontal  (between  systems)  trade-offs.  Figure  4.02 
depicts  horizontal  and  vertical  trade-offs  for  a  typical  space  system.  Figure  4.03  depicts 
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the  systems  engineering  complexity  between  two  systems  that  must  be  interoperable  to 
provide  a  specified  capability. 

Figure  4.02  -  Horizontal  and  Vertical  Trade-offs  for  a  Space  System  (Gamache,  2006) 
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At  this  time,  these  trade-offs  are  made  largely  by  both  formal  and  informal 
committees,  although  the  committee  membership  differed  amongst  interviewee 
responses.  Program  offices  and  SMC  at  large,  have  chief  system  engineers  that  monitor 
these  trade-offs  and  provide  recommendations,  but  an  overarching  capability  has  no  chief 
systems  engineer  or  director  responsible  for  making  such  decisions  across  platforms  and 
services.  In  addition  to  systems  engineers  trained  at  the  system,  program,  and  capability 
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levels,  DoD  acquisitions  could  benefit  from  switching  service-specific  Program 
Executive  Officers  (PEO)  to  joint  Capability  Acquisition  Executives  (CAE).  This  would 
reduce  redundancy,  excessive  levels  of  oversight  and  burden  on  the  J-8,  and  avoid  trade¬ 
off  decisions  being  made  by  what  one  interviewee  described  as  a  room  full  of  “500-lb 
gorillas.” 

Figure  4. 03  -  Horizontal  and  Vertical  Trade-offs  for  Interoperable  Space  Systems 
(Gamache,  2006) 
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What  are  the  goals  of  capability-based  acquisitions?  The  main  goal  of  capability- 


based  acquisitions  is  to  develop  acquisition  solutions  that  are  responsive  to  the  way  DoD 


conducts  operations.  Specifically,  reductions  in  funding  and  changes  in  threats  require 
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more  agile  and  efficient  warlighting,  which  can  be  accomplished  by  eliminating  un¬ 
needed  redundancy  of  efforts  and  people  and  supplementing  with  easier  information 
exchange.  Capability-based  acquisitions  accomplishes  this  by  encouraging  solutions  to 
be  joint,  interoperable,  and  netcentric  and  emphasizing  the  development  of  a  capability 
over  the  platform. 

How  are  capability-based  acquisitions  implemented?  JCIDS’  guidance  primarily 
addresses  JCIDS  oversight,  review,  and  documentation  of  the  decision  support  system. 
There  is  little  to  no  appropriate  implementation  guidance  for  the  program  level.  Aside 
from  there  not  being  DoD-directed  acquisition  policy,  Air  Force  Instruction  63-1 
“Capability-Based  Acquisition  System”  (2003)  does  not  give  any  guidance  on 
implementation,  let  alone  define  the  tenn.  “Joint”  is  used  only  twice  in  the  definition  for 
warfighter  and  in  referencing  a  document.  AFI  63-1  also  specifically  states  that  it  does 
not  apply  to  space  systems. 

How  does  the  acquisition  workforce  view  the  implementation  of  JCIDS  and 
NSSAP  documentation?  None  of  the  interviewees  experienced  or  expected  to  experience 
a  reduction  in  capability  development  time  or  cost  overruns.  Several  reasons  were  cited 
by  the  interviewees  including: 

•  Too  much  technology  development  after  committing  to  a  solution 

•  Funding  profile  not  matching  identified  space  system  profile  of  80%  in 
development  and  production  and  20%  in  operations  and  maintenance 

•  Unstable  funding  or  not  enough  reserve  (25%)  for  Program  Manager  flexibility 

•  No  day-to-day  operations  change  in  JCIDS  from  RGS,  other  than  joint  focus 
which  does  not  trickle  down  through  Service 
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4.3  Recommendations 


The  results  and  discussion  of  the  literature  review,  interviews,  and  investigative 
questions  led  to  recommendations  in  the  categories  of: 

•  Acquisition  Expertise 

•  Requirements  Control 

•  Capable  Technology  and  Industrial  Base 

•  Realistic  Budgeting 

Some  recommendations  are  applicable  to  system  acquisition  outside  of  the  space 
community.  Although  many  of  these  recommendations  were  included  in  the  discussion 
section,  they  are  consolidated  in  the  subsequent  sections. 

4.3.1  Acquisition  Expertise 

Service  PEOs  need  to  be  converted  into  Joint  Capability  Acquisition  Executives 
(CAEs),  responsible  for  acquiring  a  capability  through  the  management  of  a  portfolio  of 
programs.  Joint  CAEs  would  be  responsible  for  ensuring  executable  APBs  for 
supporting  programs  and  PMs  must  have  adequate  pre-acquisition  funding  to  assemble 
people,  tools  and  data  to  establish  executable  APBs  with  the  CAE.  Expertise  could  be 
cultivated  by  requiring  a  rotational  assignment  in  industry  and  systems  engineering 
before  becoming  a  PM.  Level  three  space  professional  certification  would  be  a  requisite 
for  all  PM  and  CAE  positions  for  space  systems. 

4.3.2  Requirements  Control 

Controlling  requirements  is  necessary  for  implementing  an  evolutionary 
acquisition  strategy,  but  even  more  critical  with  the  acquisition  of  capabilities  through 
SoSs  and  FoSs.  The  Association  for  Operations  Management  recommends  limiting  key 
management  requirements  to  somewhere  between  three  and  seven.  With  the  help  of  the 


67 


users,  CAEs  must  establish  or  accept  no  more  than  five  key  performance  parameters 
(KPPs)  per  system  supporting  the  overall  capability,  for  example.  This  will  allow 
program  managers  (PM)  to  have  more  degrees  of  freedom  in  how  their  portion  of 
capability  is  provided.  Similarly,  with  the  help  of  the  users,  PMs  should  further  limit  the 
number  of  KPPs  per  block.  It  is  true  that  changes  in  threats,  technology,  or  available 
knowledge  may  require  an  additional  validated  KPP,  however,  limiting  KPPs  per  system 
or  increment  will  force  system  development  to  move  forward  and  use  following 
increments  as  they  were  meant  to  be  used  to  shorten  cycle  time  and  account  for  these 
changes. 

Requirements  control  can  also  be  accomplished  at  the  capability-level.  As  the 
expert  on  currently  available  capability,  CAEs  would  be  responsible  for  completing 
Analyses  of  Alternatives  (AoAs)  instead  of  Service-specific  requirements  organizations 
working  with  a  System  Program  Office.  To  accomplish  this,  CAEs  with  system-of- 
systems  engineering  (SoSE)  expertise  would  need  to  invest  in  accredited  SoSE  tools  and 
SoS  engineers  with  JCIDS  and  NSSAP  Joint  Program  Office  experience.  Finally,  the 
CAE  should  establish  SoS  measures  of  effectiveness  (MOEs).  After  ah,  “it  is  important 
to  understand  that  ‘metrics  matter.’  We  must  be  able  to  define  success  and  measure  it.  On 
the  battlefield,  we  can  easily  measure  the  perfonnance  of  the  capabilities  provided 
through  the  Global  Positioning  System  or  Military  Satellite  Communication.  We  need  to 
do  the  same  as  we  develop  the  next  generation  space  capabilities.  We  must  know  where, 
when  and  how  we  are  succeeding... and  failing”  (Lord,  2006). 
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4.3.3  Capable  Technology  &  Industrial  Base 


The  commercial  market  for  space  has  been  rather  unpredictable.  With  the 
consolidation  of  defense  space  contractors,  the  space  system  community  needs  a  better 
mechanism  to  “pull”  DoD  science  and  technology  investments  and  to  provide  guidance  to 
the  industrial  base  for  independent  research  and  development  (IRAD)  and  capital 
investment,  ensuring  that  technology  matures  at  a  steady  or  accelerated  rate. 

4.3.4  Realistic  Budgeting  and  Funding 

Budget  instability  was  a  major  concern  among  interviewees.  Two 
recommendations  can  be  made  based  on  two  main  causes  of  inaccurate  budgets  being  not 
enough  information  or  designing  to  not  enough  money. 

The  space  system  community  needs  a  methodology  for  creating  more  accurate 
cost  estimates  of  Horizontal  Integration  (HI),  immature  technologies,  and  space  industrial 
base  restructuring  costs.  Additionally,  the  Young  Panel  recommendation  of  an  80/20 
ratio  of  funds  with  a  25%  margin  and  not  using  contractor  estimates  in  competitive 
situations  would  increase  the  accuracy  of  initial  cost  estimates.  Requiring  that  SPO  cost 
estimates  and  Independent  Cost  Estimates  reconcile  at  KDPs  would  provide  insight  into 
problem  areas  and  reduce  the  frequency  and  severity  of  unforeseen  overrun. 

Although  not  adequately  examined  due  to  the  scope  of  the  research,  one 
recommendation  could  be  made  for  joint  systems,  such  as  space  systems  and  information 
systems,  being  funded  out  of  separate  joint  funding,  as  opposed  to  service-specific 
funding. 
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4.4  Summary 


Chapter  Four  summarized  the  results  of  the  literature  review  and  interviews 
analysis  with  respect  to  the  investigative  questions.  Due  to  restrictions  placed  by  the 
Human  Research  Board  the  names,  organizations,  and  specific  quotations  could  not  be 
included  in  the  published  thesis.  The  synthesized  data  identified  four  areas  of 
recommendations  that  would  lead  to  the  improvement  of  the  interaction  between  the 
JCIDS  and  NSSAP. 
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V.  Conclusion 


5.1  Introduction 

The  results  and  recommendations  presented  in  Chapter  Four  lay  the  framework 
for  a  theory  to  be  presented.  DoD’s  challenge  is  to  create  an  environment  that  enables 
program  managers’  mission  success  by  ensuring  that  the  acquisition  program  baselines 
for  all  National  Security  Space  systems  are  executable  and  stable.  That  environment  lies 
within  the  overlap  of  the  JCIDS,  NSSAP,  and  PPBE  Process. 

5.2  Theory  Building 

Once  in  that  environment,  a  program  manager  will  be  faced  with  many  decisions 
that  must  be  made  considering  six  factors,  or  trade  space  areas.  Those  areas  for  trade 
space  are:  requirements,  cost,  schedule,  risk,  system  concept,  and  the  available 
technology  and  industrial  base.  Figure  5.01  illustrates  this  concept  with  the  margin  for 
trade  in  the  middle. 

Previous  models  have  presented  a  similar  decision  making  box,  typically 
considering  cost,  schedule,  and  perfonnance  (or  requirements).  However,  research  shows 
that  many  more  factors  play  a  role  in  determining  mission  success  for  space  systems. 
The  employment  of  an  evolutionary  strategy  in  the  NSSAP  attempts  to  make  use  of  the 
most  current,  mature  technologies  and  mitigate  risk,  in  addition  to  lowering  cost  through 
risk  reduction.  “Pursuit  of  leap-ahead  capabilities  will  need  to  accommodate  risk 
reduction  and  technology  maturation,  most  likely  through  spiral  development  and 
evolutionary  progress.”  (Neuman,  2006)  Similarly,  the  change  to  capability-based 
acquisitions  in  the  JCIDS,  not  only  affects  requirement  generation,  but  also  opens  the 
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door  for  employing  system-of-systems  (SoSs)  to  meet  a  capability  deficiency,  requiring 
attention  be  paid  to  system  concept. 


Figure  5. 01  -  The  Program  Manager ’s  Decision  Making  Box  (Gamache,  2006) 
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“Like  any  policies,  how  you  deal  with  them  is  key”  (Coyle,  2003)  and  the  same  is 
true  for  the  CJCS  series  and  NSSAP  03-01.  While  the  policies  are  fairly  sound,  with  a 
number  of  good  ideas  like  NSSAP  03-0  l’s  independent  program  assessment  (IP  A),  the 
tools,  mechanisms,  and  people  are  not  in  place  to  implement  the  policies  with  their 
desired  intent. 
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The  recommendations  presented  in  Chapter  Four  state  some  of  the  tools  and  how 
to  acquire  the  people  needed  to  implement  such  policies,  enabling  better  interaction 
between  JCIDS  and  NS  SAP. 

5.3  Future  Research 

Additional  research  can  be  conducted  in  a  number  of  areas.  Because  JCIDS 
applies  to  all  services  and  a  number  of  the  recommendations  would  affect  all  services, 
similar  research  should  be  accomplished  within  the  sister  services  before  attempting  to 
correct  any  problems  with  the  JCIDS.  The  concept  of  joint  Capability  Acquisition 
Executives  managing  architectures  of  SoSs  and  FoSs  must  be  further  developed. 

As  mentioned  in  Chapter  One,  a  limitation  of  the  research  was  that  it  did  not 
address  the  PPBE  Process.  Budget  stability  was  a  major  theme  in  the  interviews  and  to  a 
lesser  extent  in  the  literature;  however,  it  was  not  examined  in  this  research  to  keep  the 
scope  manageable.  The  PPBE  Process  should  also  be  reviewed  for  space  systems. 

In  the  author’s  opinion,  one  of  the  most  important  areas  of  future  research  is  in 
DoD’s  implementation  practices.  Specifically,  one  could  research  whether  tools, 
mechanisms,  and  skilled  people  were  in  place  before  or  after  a  policy  was  released. 
DoD’s  philosophy  on  the  role  of  policies  could  also  be  defined. 

Finally,  the  recommendations  of  this  thesis  should  be  tested  before  implemented. 
While  literature  and  interview  data  support  the  recommendations,  a  more  thorough 
investigation  as  to  the  implications  of  such  recommendations  should  be  made  before 
implementation. 
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Appendix  A 

Interview  Invitation  Letter 


Dear  (Interview  Participant’s  Name), 

I  am  a  graduate  student  at  the  Air  Force  Institute  of  Technology,  studying 
Research  and  Development  Management  and  Space  Systems.  My  master’s  thesis 
research  is  examining  the  interaction  between  the  Joint  Capability  Integration  and 
Development  System  (JCIDS)  and  the  Defense  Acquisition  System  (DAS).  As  your 
schedule  permits,  I  would  like  to  set  up  an  appointment  to  interview  you  about  your 
experiences  on  the  subject. 

My  research  includes  interviews  with  people  throughout  the  acquisition  process 
for  developing  the  nation’s  space  systems,  from  the  individuals  operating  the  resultant 
system  to  those  individuals  who  identified  the  existing  deficiency  requiring  such  a  system 
to  be  developed.  This  research  will  describe  the  degree  of  interaction  between  the  JCIDS 
and  the  DAS,  identify  major  themes,  and  propose  recommendations  for  areas  of 
improvement  as  these  areas  are  discovered. 

The  interview  will  follow,  but  is  not  limited  to,  a  standard  set  of  questions  that  are 
included.  This  semi-structured  approach  will  allow  for  the  analysis  of  responses  to  same 
questions  across  all  interviews,  as  well  as  provide  for  any  additional  information  that  may 
be  helpful  to  understanding  an  organization’s  perspective  or  provide  clarity.  Information 
collected  during  the  interview  will  remain  confidential  throughout  the  entire  process, 
including  the  resulting  thesis  and  related  articles.  The  interview  questions  have  been 
approved  through  the  Air  Force  Institute  of  Technology’s  process  for  Human  Subject 
Research  Approval. 

Please  respond,  indicating  whether  you  are  or  are  not  willing  to  participate  in  such 
an  interview.  If  willing,  also  indicate  a  time  and  date  convenient  to  you  and  falling  on  or 
before  3 1  Jan  2006  for  the  interview  to  occur.  With  your  permission,  the  interview  will 
be  recorded.  The  interview  may  also  be  administered  through  an  e-mail  dialogue  if  you 
prefer. 


Thank  you  in  advance  for  your  time,  consideration,  and  willingness  to  participate 
in  my  thesis  research.  If  you  have  any  questions,  I  can  be  reached  by  phone  at  937-xxx- 
xxxx  and  by  e-mail  atjoyce.gamache@afit.edu. 


Sincerely, 

Joyce  Gamache 
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Appendix  B 
Survey  Questions 


Name  and  Rank: 

Duty  Title  and  Description: 

Relevant  Coursework  (DSMC,  DAU)  or  Seminars: 

Interview  Time  and  Method: 

Instructions 

Y ou  are  reminded  that  the  personal  information  and  responses  collected  today 
will  remain  completely  confidential.  They  are  collected  to  develop  themes  and  measure 
the  effectiveness  of  the  implementation  of  the  CJCSI  3170  and  NSSAP  03-01  on  the  Joint 
Capability  Integrated  Development  System  (JCIDS)  and  the  Defense  Acquisition  System 
(DAS). 

With  your  permission,  this  interview  will  be  recorded  by  digital  voice  recorder. 

Y ou  will  be  asked  the  eight  standard  questions  that  were  e-mailed  to  you  earlier  and  may 
be  asked  additional  participant-specific,  follow-up  questions.  Please  answer  the  seven 
questions  with  respect  to  GPS,  SBIRS,  TSAT,  and  any  other  relevant  space  program  that 
you  have  had  experience  with.  Thank  you  for  participating  in  this  research. 

Standard  Questions 

1.  Please  describe  your  organization’s  role  in  the  JCIDS  &  DAS  processes. 

2.  The  NSSAP  03-01  defines  evolutionary  acquisitions,  spiral  development  and 
incremental  development  as  follows. 

EA  is  defined  as  an  acquisition  approach  that  delivers  capability  in 
increments,  recognizing  up  front  the  need  for  future  capability 
improvements.  This  approach  requires  collaboration  among  the  user, 
tester,  and  developer.  The  two  main  processes  to  perform  EA  are: 

a)  Spiral  Development.  In  this  process,  a  desired  capability  is 
identified,  but  the  end-state  requirements  are  not  known  at  program 
initiation.  Those  requirements  are  refined  through  demonstration  and  risk 
management,  there  is  continuous  user  feedback,  and  each  increment 
provides  the  user  the  best  possible  capability.  The  requirements  for  future 
increments  depend  on  feedback  from  users  and  technology  maturation. 

b)  Incremental  Development.  In  this  process,  a  desired  capability  is 
identified,  an  end- state  requirement  is  known,  and  that  requirement  is  met 
over  time  by  development  of  several  increments,  each  dependent  on 
available  mature  technology. 

Do  the  following  programs  employ  an  evolutionary  acquisition  (EA)  strategy?  If  so,  does 
the  program  use  incremental  or  spiral  development  and  why  was  this  method  selected 
over  the  alternative? 

GPS  - 
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SBIRS  - 
TSAT- 
Other  - 

3.  Who  is  responsible,  and  what  processes  or  tools  do  they  use,  for  making  trade-offs 
between  the  capabilities  provided  by  the  system-of-systems  (SoS)  or  family-of-systems 
(FoS)  at  the  mission  level,  and  then  deciding  the  space  system  acquisition  spirals  or 
increments?  What  sort  of  documentation  is  required  (if  any)  to  reflect  this? 

GPS  - 
SBIRS  - 
TSAT- 
Other  - 

4.  Has,  or  will,  the  NSSAP  03-01  implementation  of  EA  on  space  programs  dramatically 
reduced  capability  timelines  and  space  program  overruns?  Why  or  why  not? 

GPS  - 
SBIRS  - 
TSAT- 
Other  - 

5.  Has,  or  will,  the  CJCSI  3170  implementation  of  the  JCIDS  process  dramatically 
reduced  capability  timelines  and  space  program  overruns?  Why  or  why  not? 

GPS  - 

SBIRS  - 
TSAT- 
Other  - 

6.  Prior  to  the  implementation  of  EA  through  DoDD  5000.1  and  DoDI  5000.2  for  the 
DAS  and  NSSAP  03-01  for  space  systems,  what  kind  of  acquisition  process  would  have 
been  undertaken? 

GPS  - 
SBIRS  - 
TSAT- 
Other  - 

7.  Please  review  the  diagrams  of  CJCSI  3170  and  NSSAP  03-01  products  and  events  as  a 
function  of  major  program  phase.  Please  identify  what  has  worked  well  and  why. 

GPS  - 
SBIRS  - 
TSAT- 
Other  - 

8.  Please  review  the  diagrams  of  CJCSI  3170  and  NSSAP  03-01  products  and  events  as  a 
function  of  major  program  phase.  Please  identify  what  has  not  worked  well  and  why. 
GPS  - 

SBIRS  - 
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Figure  3.1  -  CJCSI  3170  Products  and  Events  as  a  Function  of  Major  Phase  (CJCSI 
3170,2005) 
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Figure  3.2  -  NSSAP  03-01  Products  and  Events  as  a  Function  of  Major  Program  Phase, 
Small  Quantity  System  Model  (NSSAP  03-01,  2004) 
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Figure  3.3  -  NSSAP  03-01  Products  and  Events  as  a  Function  of  Major  Program  Phase, 
Large  Quantity  System  Model  (NSSAP  03-01,  2004) 


Large  Quantity  Production  Focused  Model  —  NSS  Acquisition  Policy  03-01 


Pre-Sys  terns  Acquisition 


Systems  Acquisition 


Key 

Decision 

Points: 


PHASE  A 
Approval 


PHASE  B 
Approval 


PHASE  C 


/XOCVV 
ICD  V 


Initial, 

CDD 


Meeting 


JROC, 

CDD 


LRIP 

Decisioi 


JROCQ 


Sustainment 


Full  Rate 
Pr  odor  boa 
Decision 

IOC  <Q>  FOC 


Phase  B 
Preliminary 
Design 


4$<>0 

Concept 

Dec  won  ^  ^ 


79 


Bibliography 


1 .  Addelston,  Jonathan  D.  Conference  on  the  Acquisition  of  Software-Intensive  Systems. 

“System-of-Systems  Architecture  and  TSPR  Contractor  Model.”  29  Jan  2003. 

2.  Air  Force  Materiel  Command.  Pamphlet  63-X  (Draft):  Evolutionary  Acquisition 

Strategies.  20  Dec  2005. 

3.  Air  Force  Space  Command.  Strategic  Master  Plan  FY06  and  Beyond.  1  Oct  2003. 

4.  Aldridge,  E.C.  Jr.  Memorandum  of  the  Under  Secretary  of  Defense.  “Evolutionary 

Acquisition  and  Spiral  Development.”  12  Apr  2002. 

5.  Bjorkman,  Eileen.  “Joint  Test  and  Evaluation  Methodology:  Methods  and  Processes 

for  Testing  in  a  Joint  Environment.”  9  Nov  2005. 

6.  Boehm,  Barry.  Crosstalk.  “The  Spiral  Model  as  a  Tool  for  Evolutionary 

Acquisition.”  May  2001 

7.  Boehm,  Barry  and  Richard  Turner.  Balancing  Agility  and  Discipline.  Addison- 

Wesley.  Boston,  MA.  2004. 

8.  Boehm,  Barry.  Ground  System  Architectures  Workshop.  “Spiral  Acquisition  of 

Defense  and  Space  System  of  Systems.”  1  Apr  2004. 

9.  Boehm,  Barry.  Space  System  Engineering  &  Acquisition  Excellence  Forum. 

“Challenges  in  Integrating  Contractual  and  Technical  Acquisition  Practices.”  4 
Aug  2005. 

10.  Buede,  Dennis  M.  The  Engineering  Design  of  Svsterns.  John  Wiley  &  Sons,  Inc. 

New  York,  NY.  2000. 

1 1 .  Brown,  David  P.  Space  System  Engineering  &  Acquisition  Excellence  Forum. 

“Current  Directions  in  Evolutionary  Acquisition.”  4  Aug  2005. 

12.  Chairman  of  the  Joint  Chiefs  of  Staff.  Instruction  3 170.0 IE :  Joint  Capabilities 

Integration  and  Development  System.  1 1  May  2005. 

13.  Chairman  of  the  Joint  Chiefs  of  Staff.  Instruction  3 170.0  IB:  Requirements 

Generation  System.  1 5  Apr  200 1 . 

14.  Coyle,  Philip  E.  III.  Program  Manager.  “Evolutionary  Acquisition:  Seven  Ways  to 

Know  if  You  Are  Placing  Your  Program  at  Unnecessary  Risk.”  Nov-Dee  2000. 
P.2-5 


80 


15.  Creswell,  J.W.  Research  Design:  Qualitative  and  Quantitative  Approaches . 

Thousand  Oaks,  CA:  Sage.  1994. 

16.  Creswell,  J.W.  Qualitative  Inquiry  and  Research  Design:  Choosing  Among  Five 

Traditions.  Sage.  Thousand  Oaks,  CA.  1997. 

17.  Deming,  W.  Edwards.  Out  of  the  Crisis.  Massachusetts  Institute  of  Technology. 

Cambridge,  MA.  1986. 

18.  Department  of  Defense.  Directive  5000.1:  Defense  Acquisition.  15  Mar  1996. 

19.  Department  of  Defense.  Instruction  5000.2:  Operation  of  the  Defense  Acquisition 

System.  12  May  2003. 

20.  Department  of  Defense.  Military  Standard  499-C  (Draft):  Systems  Engineering.  24 

Mar  2005. 

21.  Department  of  Defense.  National  Security  Space  Acquisition  Policy  03-01:  Guidance 

for  DoD  Space  System  Acquisition  Process.  27  Dec  2004. 

22.  Department  of  Defense,  Office  of  Force  Transfonnation.  “Top  Five  Goals  of  the 

Director,  Force  Transformation.”  Accessed  04  Feb  2006. 
http://www.oft.osd.mil/top_five_goals.cfim 

23.  Department  of  Defense.  Report  of  the  Defense  Science  Board  Task  Force  on  Defense 

Acquisition  Reform.  Jul  1993. 

24.  Department  of  Defense.  Report  of  the  Defense  Science  Board  Task  Force  on  Defense 

Acquisition  Reform,  Phase  II.  Aug  1994. 

25.  Department  of  Defense.  Report  of  the  Defense  Science  Board  Task  Force  on 

Acquisition  Reform  Phase  IV:  Subpanel  on  Research  and  Development.  Jul  1999. 

26.  Department  of  Defense.  Report  of  the  Defense  Science  Board  Task  Force  on  Space 

and  Missile  Tracking  System.  Aug  1996. 

27.  Department  of  Defense.  Report  of  the  Defense  Science  Board/Air  Force  Scientific 

Advisory  Board  Joint  Task  Force  on  Acquisition  of  National  Security  Space 
Programs.  May  2003. 

28.  Department  of  Defense.  Transformation  Planning  Guidance.  Apr  2003. 

29.  Eisenhardt,  K.M.  Building  Theories  from  Case  Studv  Research.  Thousand  Oaks,  CA: 

Sage.  1989. 


81 


30.  Fabey,  Michael.  Defense  News.  “DoD  Plan  Calls  for  Taking  GPS  Away  from 

USAF.”  16  Jan  2006. 

31.  Farkas,  Kenneth.  Program  Manager.  “Evolutionary  Acquisition  Strategies  and 

Spiral  Development  Processes:  Delivering  Affordable,  Sustainable  Capability  to 
the  Warfighters.”  Jul-Aug2003. 

32.  Farkas,  Kenneth  and  Michael  Fanner.  Systems  249:  Evolutionary  Acquisitions 

Strategies  and  Spiral  Development  Processes.  28  Mar  2005. 

33.  Fanell,  Lawrence  B.  Jr.  NDIA.  “Evolving  Themes  in  Defense  Transformation.”  Jun 

2002. 

34.  Garamone,  Jim.  Defenselink.  “Senate  grills  Myers  on  Homeland  Defense, 

Transformation.”  14  Sep  2001  (Accessed  04  Feb  2006). 
http://www.defenselink.mil/news/Sep200  l/n09 14200 1_200 1 09 14 1 0.html 

35.  Gettle,  Mitch.  Air  Force  Link.  “Moseley  Confirmation  Hearing  Held.”  30  Jun  2005. 

http://www.af.mil/news/story.asp7storyIDM23010927 

36.  General  Accounting  Office.  Defense  Acquisitions:  Improvements  Needed  in  Space 

Systems  Acquisition  Management  Policy,  GAO-03-1073.  Sep  2003. 

37.  General  Accounting  Office.  Military  Space  Operations:  Planning,  Funding,  and 

Acquisition  Challenges  Facing  Efforts  to  Strengthen  Space  Control,  GAO-02-738. 
Sep  2002. 

38.  Goulet,  Christopher  J.  “Evolutionary  Acquisition  Pamphlet.”  20  Dec  2005. 

39.  Government  Accountability  Office.  Space  Acquisitions:  Stronger  Development 

Practices  and  Investment  Planning  Needed  to  Address  Continuing  Problems, 
GAO-05-891T.  Jul2005. 

40.  Hamel,  Michael  A.  High  Frontier.  “Military  Space  Acquisition:  Back  to  the  Future.” 

Vol  2.  Num  2.  Jan  2006.  P.4-7. 

41.  Hawthorn,  Skip  and  Ramona  Lush.  Crosstalk.  ‘Evolutionary  Acquisition  and  Spiral 

Development.  ”  Aug  2002. 

42.  Johnson,  Collie  J.  Program  Manager.  “Pentagon  Systems  Acquisition  Director  [John 

C.  Wilson]  Speaks  to  Graduates  of  APMC  99-1.”  May- Jun  1999.  P  8-1 1. 

43.  Joint  Staff  /  J8.  “JCIDS  Overview  for  N1PA.”  3  Nov  2005. 

44.  Johnson,  Wayne  M.  and  Carl  O.  Johnson.  Acquisition  Review  Quarterly.  “The 

Promise  and  Perils  of  Spiral  Acquisition.”  Summer  2002.  P.  174-188. 


82 


45.  Larman,  Craig  and  Victor  Basili.  IEEE.  “Iterative  and  Incremental  Development:  A 

Brief  History.”  Jun2003.  P.47-56. 

46.  Lee,  Douglas  E.  Air  and  Space  Power  Journal.  “Space  Reform.”  Summer  2004. 

47.  Levin,  Robert  E.  High  Frontier.  “Overcoming  Space  Acquisition  Problems.”  Vol  2. 

Num  2.  Jan  2006.  P.14-17. 

48.  Lord,  Lance  W.  High  Frontier.  “People,  Process,  and  Partnerships. .  .Our  Keys  to 

Acquisition  Success!”  Vol  2.  Num  2.  Jan  2006.  P.2-3. 

49.  Lumb,  Mark  D.  2nd Annual DAU-South  Contracting  Conference  &  Expo.  “DoD 

Business  Transfonnation  &  the  5000  Series  Regulations:  Meeting  the  Security 
Challenges  of  the  21st  Century.”  18  &  19  Feb  2004. 

50.  MacConnack,  Alan.  MIT  Sloan  Management  Review.  “Product-Development 

Practices  that  Work:  How  Internet  Companies  Build  Software.”  Winter  2001. 

5 1 .  McKinney,  Richard  W.  High  Frontier.  “Space  Acquisition  Today.”  Vol  2.  Num  2. 

Jan  2006.  P.20-2 1. 

52.  McNulty,  James  P.  Defense  AT&L.  “Space:  The  Ultimate  High  Ground.”  Jul-Aug 

2005.  P.2-10. 

53.  Merriam,  S.B.  Case  Study  Research  in  Education:  A  Qualitative  Approach.  San 

Francisco,  CA:  Jossey-Bass.  1988. 

54.  Miles,  M.B.  &  Huberman,  A.M.  Qualitative  Data  Analysis,  Second  Edition. 

Thousand  Oaks,  CA:  Sage.  1994. 

55.  Morse,  J.M.  “Approaches  to  Qualitative-Quantitative  Methodological 

Triangulation.”  Nursing  Research,  40(1),  120-123. 

56.  Myers,  Fred.  NDIA  Systems  Engineering  Conference.  “Improving  M&S  Support  to 

Acquisition:  A  Progress  Report  on  Development  of  the  Acquisition  M&S  Master 
Plan.”  26  Oct  2005. 

57.  National  Defense  Industrial  Association.  Study  Task  Report:  M&S  Support  to  the 

New  DoD  Acquisition  Process.  Feb  2004. 

58.  Neuman,  Kurt  M.  High  Frontier.  “Restoring  Credibility:  The  Road  to  Space 

Acquisition  Recovery.”  Vol  2.  Num  2.  Jan  2006.  P.47-50. 

59.  Pawlikowski,  Ellen  M.  High  Frontier.  “Moving  Beyond  Acquisition  Reform’s 

Fegacy.”  Vol  2.  Num  2.  Jan  2006.  P.27-32. 


83 


60.  President’s  Blue  Ribbon  Commission  on  Defense  Management.  A  Quest  for 

Excellence.  “Executive  Summary.”  Jun  1986. 

61.  Rich,  Mary  A.,  Steve  Lazar,  and  Joe  Betser.  Ground  System  Architectures  Workshop. 

“Space  Acquisition  Strategy  -  Just  How  Important  is  the  Ground  Segment?”  4-6 
Mar  2003. 

62.  Rippere,  Richard  B.  Defense  AT&L.  “Acquisition  Transformation:  Lead  into  Gold?” 

Jul-Aug  2004.  P.36-39. 

63.  Roberts,  Katherine.  High  Frontier.  “’A  Crisis  Situation’:  Danger  and  the  Crucial 

Point.”  Vol  2.  Num  2.  Jan  2006.  P.22-26. 

64.  Rounce,  Scott  C.  The  Commercial  Revolution  in  Space  Acquisition  Management. 

Industrial  College  of  the  Armed  Lorces.  1996. 

65.  Secretary  of  the  Air  Lorce.  Instruction  10-601:  Capabilities  Based  Requirements 

Development.  30  Jul  2004. 

66.  Secretary  of  the  Air  Lorce.  Instruction  63-101:  Operations  of  Capabilities  Based 

Acquisition  System.  29  Jul  2005. 

67.  Secretary  of  the  Air  Lorce.  Instruction  63-123:  Evolutionary  Acquisition  for  C2 

Systems.  1  Apr  2000. 

68.  Secretary  of  the  Air  Lorce.  Policy  Directive  63-1 :  Capability-Based  Acquisition 

System.  10  Jul  2003. 

69.  Sheridan,  John  T.  “Tom.”  High  Frontier.  “Today’s  National  Security  Space 

Acquisition  Environment:  Learning  from  the  Past  -  A  Path  Lorward.”  Vol  2. 

Num  2.  Jan  2006.  P.18-19. 

70.  SMC  ACE.  Space  System  Engineering  &  Acquisition  Excellence  Forum. 

“Integrating  Business  and  Technical  Perspectives  into  the  Acquisition  Process.”  4 
Aug  2005. 

71.  Stevens,  Robert  J.  High  Frontier.  “Successes  and  Challenges  Pacing  the  Acquisition 

System.”  Vol  2.  Num  2.  Jan  2006.  P.8-9 

72.  Sugar,  Ronald  J.  High  Frontier.  “The  Three-Body  Problem:  Perspective  on  Space 

Acquisition.”  Vol  2.  Num  2.  Jan  2006.  P.10-13. 

73.  Sylvester,  Richard  K.  and  Joseph  A.  Perrara.  Acquisition  Review  Quarterly. 

“Conflict  and  Ambiguity:  Implementing  Evolutionary  Acquisition.”  Winter  2003. 
P.3-27. 


84 


74.  Teets,  Peter  B.  Memorandum  of  the  Under  Secretary  of  the  Air  Force.  “National 

Security  Space  (NSS)  Acquisition  Policy  03-01.”  6  Oct  2003. 

75.  Tesch,  R.  Qualitative  Research:  Analysis  Types  and  Software  Tools.  New  York, 

NY:  F aimer.  1990. 

76.  Undersecretary  of  Defense  for  AT&L.  Acquisition  Modeling  and  Simulation  Master 

Plan  (Draft).  21  Oct  2005. 

77.  Unger,  Darian  W.  and  Steven  D.  Eppinger.  “Planning  Design  Iterations.” 

Massachusetts  Institute  of  Technology.  2002. 

78.  Unger,  Darian  W.  Product  Development  Process  Design:  Improving  Development 

Response  to  Market  Technical,  and  Regulatory  Risks.  Massachusetts  Institute  of 
Technology.  May  2003. 

79.  Wideman,  R.  Max.  Projects  and  Profits,  Special  Issue.  “Software  Development  and 

Linearity.”  March  2003. 

80.  Yin,  R.K.  Case  Study  Research:  Design  and  Methods,  Third  Edition.  Thousand 

Oaks,  CA;  Sage.  2003. 


85 


Vita 


First  Lieutenant  Joyce  A.  Gamache  graduated  from  Lake  Braddock  Secondary 
School  in  Burke,  Virginia.  She  entered  undergraduate  studies  at  the  University  of  Notre 
Dame  in  South  Bend,  Indiana  and  graduated  with  a  Bachelor  of  Science  degree  in 
Science  Business  in  May  2002. 

Her  first  assignment  was  to  the  MILSATCOM  Joint  Program  Office  (MJPO)  at 
the  Space  and  Missile  Systems  Center  (SMC),  Los  Angeles  Air  Force  Base,  California, 
where  she  served  as  liaison  to  the  program  element  monitors  (PEM)  for  Advanced 
Extremely  High  Frequency  (AEHF)  satellites,  Wideband  Gapfiller  Satellites  (WGS), 
Defense  Satellite  Communication  System  (DSCS),  and  Milstar. 

In  August  2004,  she  entered  graduate  studies  at  the  Air  Force  Institute  of 
Technology  in  two  master’s  degree  programs:  Space  Systems  and  Research  and 
Development  Management.  Upon  graduation  in  March  2006,  she  will  be  assigned  to  the 
Office  of  Space  Launch  within  the  National  Reconnaissance  Office. 


86 


REPORT  DOCUMENTATION  PAGE 

Form  Approved 

OMB  No.  074-0188 

The  public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and  maintaining 
the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  the  collection  of  information,  including  suggestions  for 
reducing  this  burden  to  Department  of  Defense,  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports  (0704-0188),  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA 
22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  an  penalty  for  failing  to  comply  with  a  collection  of  information  if  it  does  not  display  a 
currently  valid  OMB  control  number.  PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 

1 .  REPORT  DATE  2.  REPORT  TYPE 

15-03-06  Master’s  Thesis 

3.  DATES  COVERED  (From  -  To) 

Mar  2005 -Mar  2006 

4.  TITLE  AND  SUBTITLE 

Review  of  the  JCIDS  and  the  NSSAP 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

Gamache,  Joyce  A.,  First  Lieutenant,  USAF 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAMES(S)  AND  ADDRESS(S) 

Air  Force  Institute  of  Technology 

Graduate  School  of  Engineering  and  Management  (AFIT/EN) 

2950  Hobson  Street,  Building  642 

WPAFB  OH  45433-7765 

8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 

AFIT/GRD/ENS/06-01 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

N/A 

10.  SPONSOR/MONITOR’S 
ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED. 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

Space  systems  are  a  critical  enabler  of  the  net-centric  operation  warfare  (NCOW)  needed  to  achieve  victory  in  the  Global  War 
on  Terrorism  (GWOT).  The  effective  acquisition  of  affordable  systems  is  vital  to  our  National  Security  Strategy.  Space 
systems  play  an  important  role  throughout  a  wide  spectrum  of  military  and  civil  operations.  Several  challenging  factors  unique 
to  space  systems  development  are  the  high  level  of  technological  complexity,  a  broad  joint  user  base,  and  the  reliance  on 
seamless  interoperable  systems  to  achieve  superior  capabilities  for  US  warfighters.  This  research  examines  the  interaction 
between  the  Joint  Capabilities  Integration  and  Development  System  (JCIDS)  and  the  National  Security  Space  Acquisition 
Process  (NSSAP)  through  a  qualitative  case  study  and  identifies  ways  to  improve  this  interaction  by  answering  investigative 

questions  and  providing  recommendations  to  be  tested  in  future  research. _ 

15.  SUBJECT  TERMS 

Acquisition  Process,  Acquisition  Reform,  Space  Acquisition,  JCIDS,  NSSAP,  CJCSI/M  3170,  NSSAP  03-01,  Evolutionary 
Acquisition,  Capability-based  Acquisition 


16.  SECURITY  CLASSIFICATION 

17. 

18.  NUMBER 

19a.  NAME  OF  RESPONSIBLE  PERSON 

OF: 

LIMITATION 

OF 

OF  PAGES 

a.  REPORT 

U 

b.  ABSTRACT 

U 

c.  THIS 
PAGE 

U 

ABSTRACT 

UU 

98 

19b.  TELEPHONE  NUMBER  (Include  area  code) 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std.  Z39-18 


87 


88 


